[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:46:45 dist-test-slave-zh2d krb5kdc[6315](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:46:45 dist-test-slave-zh2d krb5kdc[6315](info): set up 2 sockets
Aug 06 22:46:45 dist-test-slave-zh2d krb5kdc[6315](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260806 22:46:47.015411 6305 mini_kdc.cc:121] Time spent starting KDC: real 1.272s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/krb5kdc/test-user.keytab.
Aug 06 22:46:47 dist-test-slave-zh2d krb5kdc[6315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056407, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:46:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:46:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.40.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.40.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.40.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab.
Entry for principal kudu/127.6.40.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab.
I20260806 22:46:47.139725 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.126:46879
--webserver_interface=127.6.40.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab
--principal=kudu/127.6.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:43683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.40.126:46879
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:46:47.225311 6336 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:46:47.225540 6336 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:46:47.225564 6336 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:46:47.226681 6336 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:46:47.226720 6336 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:46:47.226734 6336 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:46:47.226747 6336 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:46:47.226759 6336 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:46:47.228327 6336 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:43683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.40.126:46879
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab
--principal=kudu/127.6.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.126:46879
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6336
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:46:47.228535 6336 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:46:47.228727 6336 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:46:47.234097 6341 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:47.234108 6342 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:46:47.234143 6336 server_base.cc:1034] running on GCE node
W20260806 22:46:47.234166 6345 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:46:47.234735 6336 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:46:47.235224 6336 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:46:47.236405 6336 hybrid_clock.cc:648] HybridClock initialized: now 1786056407236391 us; error 57 us; skew 500 ppm
Aug 06 22:46:47 dist-test-slave-zh2d krb5kdc[6315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056407, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:46:47.239283 6336 init.cc:377] Logged in from keytab as kudu/127.6.40.126@KRBTEST.COM (short username kudu)
I20260806 22:46:47.239858 6336 webserver.cc:460] Webserver started at http://127.6.40.126:46199/ using document root <none> and password file <none>
I20260806 22:46:47.240126 6336 fs_manager.cc:359] Metadata directory not provided
I20260806 22:46:47.240208 6336 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:46:47.240391 6336 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:46:47.241775 6336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/minicluster-data/master-0/data/instance:
uuid: "c5d31d8c98bc41169bccdbf2b2468683"
format_stamp: "Formatted at 2026-08-06 22:46:47 on dist-test-slave-zh2d"
server_key: "57cb7216a58edeb2d8c3f2f5d9eb12b0"
server_key_iv: "bf886bcea01f0304efbc75fc8b3e0a8f"
server_key_version: "encryptionkey@0"
I20260806 22:46:47.242238 6336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/minicluster-data/master-0/wal/instance:
uuid: "c5d31d8c98bc41169bccdbf2b2468683"
format_stamp: "Formatted at 2026-08-06 22:46:47 on dist-test-slave-zh2d"
server_key: "57cb7216a58edeb2d8c3f2f5d9eb12b0"
server_key_iv: "bf886bcea01f0304efbc75fc8b3e0a8f"
server_key_version: "encryptionkey@0"
I20260806 22:46:47.243973 6336 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:46:47.244822 6355 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:47.245071 6336 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:46:47.245195 6336 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/minicluster-data/master-0/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/minicluster-data/master-0/wal
uuid: "c5d31d8c98bc41169bccdbf2b2468683"
format_stamp: "Formatted at 2026-08-06 22:46:47 on dist-test-slave-zh2d"
server_key: "57cb7216a58edeb2d8c3f2f5d9eb12b0"
server_key_iv: "bf886bcea01f0304efbc75fc8b3e0a8f"
server_key_version: "encryptionkey@0"
I20260806 22:46:47.245293 6336 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:46:47.262665 6336 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:46:47.264947 6336 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:46:47.265148 6336 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:46:47.270105 6336 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.126:46879
I20260806 22:46:47.270161 6431 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.126:46879 every 8 connection(s)
I20260806 22:46:47.270589 6336 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
I20260806 22:46:47.271601 6434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:46:47.274320 6434 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5d31d8c98bc41169bccdbf2b2468683: Bootstrap starting.
I20260806 22:46:47.275043 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 6336
I20260806 22:46:47.275100 6434 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c5d31d8c98bc41169bccdbf2b2468683: Neither blocks nor log segments found. Creating new log.
I20260806 22:46:47.275168 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/minicluster-data/master-0/wal/instance
I20260806 22:46:47.275424 6434 log.cc:824] T 00000000000000000000000000000000 P c5d31d8c98bc41169bccdbf2b2468683: Log is configured to *not* fsync() on all Append() calls
I20260806 22:46:47.275977 6305 external_mini_cluster.cc:1428] Setting key 7de1583c8fa4f498f2e9d8dff3c1389a
I20260806 22:46:47.276249 6434 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5d31d8c98bc41169bccdbf2b2468683: No bootstrap required, opened a new log
I20260806 22:46:47.278290 6434 raft_consensus.cc:348] T 00000000000000000000000000000000 P c5d31d8c98bc41169bccdbf2b2468683 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5d31d8c98bc41169bccdbf2b2468683" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 46879 } }
I20260806 22:46:47.278426 6434 raft_consensus.cc:374] T 00000000000000000000000000000000 P c5d31d8c98bc41169bccdbf2b2468683 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:46:47.278448 6434 raft_consensus.cc:729] T 00000000000000000000000000000000 P c5d31d8c98bc41169bccdbf2b2468683 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5d31d8c98bc41169bccdbf2b2468683, State: Initialized, Role: FOLLOWER
I20260806 22:46:47.278605 6434 consensus_queue.cc:260] T 00000000000000000000000000000000 P c5d31d8c98bc41169bccdbf2b2468683 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5d31d8c98bc41169bccdbf2b2468683" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 46879 } }
I20260806 22:46:47.278725 6434 raft_consensus.cc:388] T 00000000000000000000000000000000 P c5d31d8c98bc41169bccdbf2b2468683 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:46:47.278772 6434 raft_consensus.cc:482] T 00000000000000000000000000000000 P c5d31d8c98bc41169bccdbf2b2468683 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:46:47.278822 6434 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c5d31d8c98bc41169bccdbf2b2468683 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:46:47.279843 6434 raft_consensus.cc:504] T 00000000000000000000000000000000 P c5d31d8c98bc41169bccdbf2b2468683 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5d31d8c98bc41169bccdbf2b2468683" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 46879 } }
I20260806 22:46:47.279937 6434 leader_election.cc:302] T 00000000000000000000000000000000 P c5d31d8c98bc41169bccdbf2b2468683 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c5d31d8c98bc41169bccdbf2b2468683; no voters:
I20260806 22:46:47.280139 6434 leader_election.cc:288] T 00000000000000000000000000000000 P c5d31d8c98bc41169bccdbf2b2468683 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:46:47.280174 6439 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c5d31d8c98bc41169bccdbf2b2468683 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:46:47.280284 6439 raft_consensus.cc:686] T 00000000000000000000000000000000 P c5d31d8c98bc41169bccdbf2b2468683 [term 1 LEADER]: Becoming Leader. State: Replica: c5d31d8c98bc41169bccdbf2b2468683, State: Running, Role: LEADER
I20260806 22:46:47.280431 6434 sys_catalog.cc:564] T 00000000000000000000000000000000 P c5d31d8c98bc41169bccdbf2b2468683 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:46:47.280441 6439 consensus_queue.cc:237] T 00000000000000000000000000000000 P c5d31d8c98bc41169bccdbf2b2468683 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5d31d8c98bc41169bccdbf2b2468683" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 46879 } }
I20260806 22:46:47.280819 6439 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5d31d8c98bc41169bccdbf2b2468683 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c5d31d8c98bc41169bccdbf2b2468683. Latest consensus state: current_term: 1 leader_uuid: "c5d31d8c98bc41169bccdbf2b2468683" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5d31d8c98bc41169bccdbf2b2468683" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 46879 } } }
I20260806 22:46:47.280910 6442 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5d31d8c98bc41169bccdbf2b2468683 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c5d31d8c98bc41169bccdbf2b2468683" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5d31d8c98bc41169bccdbf2b2468683" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 46879 } } }
I20260806 22:46:47.280995 6439 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5d31d8c98bc41169bccdbf2b2468683 [sys.catalog]: This master's current role is: LEADER
I20260806 22:46:47.281096 6442 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5d31d8c98bc41169bccdbf2b2468683 [sys.catalog]: This master's current role is: LEADER
I20260806 22:46:47.281348 6449 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:46:47.281996 6449 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:46:47.284718 6449 catalog_manager.cc:1347] Generated new cluster ID: 37179f95dadb49d69c8e1e0f5eb0ea5c
I20260806 22:46:47.284799 6449 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 22:46:47 dist-test-slave-zh2d krb5kdc[6315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056407, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
I20260806 22:46:47.295707 6438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:47.277070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37538 (local address 127.6.40.126:46879)
0806 22:46:47.277265 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:47.277274 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:47.277482 (+ 208us) server_negotiation.cc:407] Connection header received
0806 22:46:47.279440 (+ 1958us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:47.279450 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:47.279521 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:47.279675 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:47.281950 (+ 2275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:47.283195 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:47.284376 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:47.285039 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:47.285155 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:47.289939 (+ 4784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:47.289943 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:47.289944 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:46:47.289955 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:47.293103 (+ 3148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:47.293654 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:47.293657 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:47.293657 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:47.293947 (+ 290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:47.294260 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:47.294260 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:47.294261 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:47.294613 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:47.294815 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:47.295084 (+ 269us) server_negotiation.cc:299] Negotiation successful
0806 22:46:47.295257 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":83,"threads_started":1}
I20260806 22:46:47.298918 6449 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:46:47.299517 6449 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:46:47.308166 6449 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c5d31d8c98bc41169bccdbf2b2468683: Generated new TSK 0
I20260806 22:46:47.308378 6449 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.40.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.40.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.40.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.65.keytab.
Entry for principal kudu/127.6.40.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.65.keytab.
I20260806 22:46:47.339464 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.65:0
--local_ip_for_outbound_sockets=127.6.40.65
--webserver_interface=127.6.40.65
--webserver_port=0
--tserver_master_addrs=127.6.40.126:46879
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.65.keytab
--principal=kudu/127.6.40.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:43683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:46:47.448124 6466 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:46:47.448413 6466 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:46:47.448459 6466 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:46:47.449975 6466 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:46:47.450047 6466 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:46:47.450093 6466 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.65
I20260806 22:46:47.451794 6466 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:43683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.65.keytab
--principal=kudu/127.6.40.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.40.65
--webserver_port=0
--tserver_master_addrs=127.6.40.126:46879
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6466
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.65
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:46:47.452073 6466 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:46:47.452316 6466 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:46:47.459215 6471 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:47.459233 6472 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:47.459233 6474 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:46:47.459640 6466 server_base.cc:1034] running on GCE node
I20260806 22:46:47.459865 6466 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:46:47.460136 6466 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:46:47.461279 6466 hybrid_clock.cc:648] HybridClock initialized: now 1786056407461259 us; error 51 us; skew 500 ppm
Aug 06 22:46:47 dist-test-slave-zh2d krb5kdc[6315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056407, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:46:47.464164 6466 init.cc:377] Logged in from keytab as kudu/127.6.40.65@KRBTEST.COM (short username kudu)
I20260806 22:46:47.464623 6466 webserver.cc:460] Webserver started at http://127.6.40.65:34799/ using document root <none> and password file <none>
I20260806 22:46:47.465049 6466 fs_manager.cc:359] Metadata directory not provided
I20260806 22:46:47.465137 6466 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:46:47.465312 6466 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:46:47.466550 6466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/minicluster-data/ts-0/data/instance:
uuid: "113949d220154cefaf497c5b9d549085"
format_stamp: "Formatted at 2026-08-06 22:46:47 on dist-test-slave-zh2d"
server_key: "46cbd3de29f7e6f6892b95bc45e187f7"
server_key_iv: "158a85fcdbfaedb0914fe8cd63b1fc0c"
server_key_version: "encryptionkey@0"
I20260806 22:46:47.467029 6466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/minicluster-data/ts-0/wal/instance:
uuid: "113949d220154cefaf497c5b9d549085"
format_stamp: "Formatted at 2026-08-06 22:46:47 on dist-test-slave-zh2d"
server_key: "46cbd3de29f7e6f6892b95bc45e187f7"
server_key_iv: "158a85fcdbfaedb0914fe8cd63b1fc0c"
server_key_version: "encryptionkey@0"
I20260806 22:46:47.468782 6466 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260806 22:46:47.469784 6484 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:47.470048 6466 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:46:47.470185 6466 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/minicluster-data/ts-0/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/minicluster-data/ts-0/wal
uuid: "113949d220154cefaf497c5b9d549085"
format_stamp: "Formatted at 2026-08-06 22:46:47 on dist-test-slave-zh2d"
server_key: "46cbd3de29f7e6f6892b95bc45e187f7"
server_key_iv: "158a85fcdbfaedb0914fe8cd63b1fc0c"
server_key_version: "encryptionkey@0"
I20260806 22:46:47.470268 6466 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:46:47.493997 6466 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:46:47.496536 6466 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:46:47.496757 6466 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:46:47.497212 6466 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:46:47.497609 6466 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:46:47.497669 6466 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:47.497705 6466 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:46:47.497753 6466 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:47.513542 6466 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.65:46741
I20260806 22:46:47.513634 6650 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.65:46741 every 8 connection(s)
I20260806 22:46:47.514030 6466 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
I20260806 22:46:47.515733 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 6466
I20260806 22:46:47.515833 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/minicluster-data/ts-0/wal/instance
I20260806 22:46:47.516005 6305 external_mini_cluster.cc:1428] Setting key 6ce1f9f403ddccdca301bf966fcbaddd
Aug 06 22:46:47 dist-test-slave-zh2d krb5kdc[6315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056407, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.65@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
I20260806 22:46:47.526777 6438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:47.515132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.65:44553 (local address 127.6.40.126:46879)
0806 22:46:47.515209 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:47.515211 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:47.515639 (+ 428us) server_negotiation.cc:407] Connection header received
0806 22:46:47.516652 (+ 1013us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:47.516654 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:47.516670 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:47.516737 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:47.518418 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:47.518944 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:47.520178 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:47.520412 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:47.520503 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:47.523445 (+ 2942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:47.523447 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:47.523448 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:46:47.523456 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:47.525303 (+ 1847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:47.525769 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:47.525770 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:47.525771 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:47.525838 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:47.526100 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:47.526101 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:47.526102 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:47.526219 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:47.526300 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:47.526581 (+ 281us) server_negotiation.cc:299] Negotiation successful
0806 22:46:47.526670 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 22:46:47.527055 6654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:47.515007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:46879 (local address 127.6.40.65:44553)
0806 22:46:47.515359 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:47.515531 (+ 172us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:47.516537 (+ 1006us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:47.516815 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:47.516819 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:47.517902 (+ 1083us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:46:47.518294 (+ 392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:47.518301 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:47.519035 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:47.519036 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:47.520075 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:47.520078 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:47.520513 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:47.520514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:47.520691 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:47.521410 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:46:47.521419 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:46:47.523312 (+ 1893us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:46:47.525439 (+ 2127us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:47.525447 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:47.525450 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:47.525675 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:47.525909 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:47.525910 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:47.525910 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:47.526020 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:47.526290 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:46:47.526298 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:46:47.526440 (+ 142us) client_negotiation.cc:769] Sending connection context
0806 22:46:47.526555 (+ 115us) client_negotiation.cc:240] Negotiation successful
0806 22:46:47.526719 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":223,"threads_started":1}
WARNING: no policy specified for kudu/127.6.40.66@KRBTEST.COM; defaulting to no policy
I20260806 22:46:47.527590 6652 heartbeater.cc:347] Connected to a master server at 127.6.40.126:46879
I20260806 22:46:47.527709 6652 heartbeater.cc:464] Registering TS with master...
I20260806 22:46:47.527912 6652 heartbeater.cc:511] Master 127.6.40.126:46879 requested a full tablet report, sending...
I20260806 22:46:47.528537 6377 ts_manager.cc:194] Registered new tserver with Master: 113949d220154cefaf497c5b9d549085 (127.6.40.65:46741)
I20260806 22:46:47.529325 6377 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.40.65@KRBTEST.COM'} at 127.6.40.65:44553
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.40.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.40.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.66.keytab.
Entry for principal kudu/127.6.40.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.66.keytab.
I20260806 22:46:47.548754 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.66:0
--local_ip_for_outbound_sockets=127.6.40.66
--webserver_interface=127.6.40.66
--webserver_port=0
--tserver_master_addrs=127.6.40.126:46879
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.66.keytab
--principal=kudu/127.6.40.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:43683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:46:47.675647 6660 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:46:47.675969 6660 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:46:47.676015 6660 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:46:47.677827 6660 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:46:47.677883 6660 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:46:47.677915 6660 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.66
I20260806 22:46:47.679536 6660 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:43683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.66.keytab
--principal=kudu/127.6.40.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.40.66
--webserver_port=0
--tserver_master_addrs=127.6.40.126:46879
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6660
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.66
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:46:47.679745 6660 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:46:47.680042 6660 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:46:47.688107 6671 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:47.688131 6669 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:47.688265 6673 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:46:47.688930 6660 server_base.cc:1034] running on GCE node
I20260806 22:46:47.689188 6660 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:46:47.689522 6660 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:46:47.690716 6660 hybrid_clock.cc:648] HybridClock initialized: now 1786056407690687 us; error 42 us; skew 500 ppm
Aug 06 22:46:47 dist-test-slave-zh2d krb5kdc[6315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056407, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:46:47.694470 6660 init.cc:377] Logged in from keytab as kudu/127.6.40.66@KRBTEST.COM (short username kudu)
I20260806 22:46:47.695142 6660 webserver.cc:460] Webserver started at http://127.6.40.66:46863/ using document root <none> and password file <none>
I20260806 22:46:47.695446 6660 fs_manager.cc:359] Metadata directory not provided
I20260806 22:46:47.695528 6660 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:46:47.695695 6660 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:46:47.697229 6660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/minicluster-data/ts-1/data/instance:
uuid: "bed586c510b346b0b846f27c5b951716"
format_stamp: "Formatted at 2026-08-06 22:46:47 on dist-test-slave-zh2d"
server_key: "83b7547efca9016f76e009e6950cf699"
server_key_iv: "79f446eb25d32ddf38d564a8852b4656"
server_key_version: "encryptionkey@0"
I20260806 22:46:47.697765 6660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/minicluster-data/ts-1/wal/instance:
uuid: "bed586c510b346b0b846f27c5b951716"
format_stamp: "Formatted at 2026-08-06 22:46:47 on dist-test-slave-zh2d"
server_key: "83b7547efca9016f76e009e6950cf699"
server_key_iv: "79f446eb25d32ddf38d564a8852b4656"
server_key_version: "encryptionkey@0"
I20260806 22:46:47.699746 6660 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20260806 22:46:47.700814 6687 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:47.701098 6660 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 22:46:47.701216 6660 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/minicluster-data/ts-1/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/minicluster-data/ts-1/wal
uuid: "bed586c510b346b0b846f27c5b951716"
format_stamp: "Formatted at 2026-08-06 22:46:47 on dist-test-slave-zh2d"
server_key: "83b7547efca9016f76e009e6950cf699"
server_key_iv: "79f446eb25d32ddf38d564a8852b4656"
server_key_version: "encryptionkey@0"
I20260806 22:46:47.701311 6660 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:46:47.711369 6660 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:46:47.714136 6660 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:46:47.714421 6660 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:46:47.714874 6660 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:46:47.715292 6660 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:46:47.715363 6660 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:47.715404 6660 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:46:47.715436 6660 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:47.732508 6660 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.66:44601
I20260806 22:46:47.732625 6839 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.66:44601 every 8 connection(s)
I20260806 22:46:47.733086 6660 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
I20260806 22:46:47.736130 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 6660
I20260806 22:46:47.736210 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/minicluster-data/ts-1/wal/instance
I20260806 22:46:47.736415 6305 external_mini_cluster.cc:1428] Setting key a99d7e54d6832b455cca23ccbf26dcb3
Aug 06 22:46:47 dist-test-slave-zh2d krb5kdc[6315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056407, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.66@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
I20260806 22:46:47.746888 6438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:47.734087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.66:34241 (local address 127.6.40.126:46879)
0806 22:46:47.734144 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:47.734147 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:47.734584 (+ 437us) server_negotiation.cc:407] Connection header received
0806 22:46:47.735429 (+ 845us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:47.735430 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:47.735441 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:47.735505 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:47.736944 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:47.737561 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:47.739657 (+ 2096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:47.740076 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:47.740173 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:47.743566 (+ 3393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:47.743568 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:47.743568 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:46:47.743575 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:47.745141 (+ 1566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:47.745754 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:47.745754 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:47.745755 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:47.745805 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:47.746121 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:47.746122 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:47.746123 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:47.746230 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:47.746286 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:47.746745 (+ 459us) server_negotiation.cc:299] Negotiation successful
0806 22:46:47.746819 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 22:46:47.747231 6843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:47.734059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:46879 (local address 127.6.40.66:34241)
0806 22:46:47.734425 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:47.734467 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:47.735329 (+ 862us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:47.735572 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:47.735576 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:47.736407 (+ 831us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:46:47.736802 (+ 395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:47.736809 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:47.737666 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:47.737668 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:47.739525 (+ 1857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:47.739529 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:47.740214 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:47.740215 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:47.740432 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:47.741169 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:46:47.741178 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:46:47.743454 (+ 2276us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:46:47.745225 (+ 1771us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:47.745232 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:47.745235 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:47.745665 (+ 430us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:47.745873 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:47.745874 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:47.745874 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:47.746047 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:47.746318 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:46:47.746325 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:46:47.746557 (+ 232us) client_negotiation.cc:769] Sending connection context
0806 22:46:47.746714 (+ 157us) client_negotiation.cc:240] Negotiation successful
0806 22:46:47.746950 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":178,"threads_started":1}
I20260806 22:46:47.747978 6840 heartbeater.cc:347] Connected to a master server at 127.6.40.126:46879
I20260806 22:46:47.748106 6840 heartbeater.cc:464] Registering TS with master...
I20260806 22:46:47.748339 6840 heartbeater.cc:511] Master 127.6.40.126:46879 requested a full tablet report, sending...
I20260806 22:46:47.748821 6377 ts_manager.cc:194] Registered new tserver with Master: bed586c510b346b0b846f27c5b951716 (127.6.40.66:44601)
WARNING: no policy specified for kudu/127.6.40.67@KRBTEST.COM; defaulting to no policy
I20260806 22:46:47.749435 6377 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.40.66@KRBTEST.COM'} at 127.6.40.66:34241
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.40.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.40.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.67.keytab.
Entry for principal kudu/127.6.40.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.67.keytab.
I20260806 22:46:47.768357 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.67:0
--local_ip_for_outbound_sockets=127.6.40.67
--webserver_interface=127.6.40.67
--webserver_port=0
--tserver_master_addrs=127.6.40.126:46879
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.67.keytab
--principal=kudu/127.6.40.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:43683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:46:47.892311 6848 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:46:47.892565 6848 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:46:47.892611 6848 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:46:47.894520 6848 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:46:47.894613 6848 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:46:47.894670 6848 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.67
I20260806 22:46:47.897089 6848 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:43683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.67.keytab
--principal=kudu/127.6.40.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.40.67
--webserver_port=0
--tserver_master_addrs=127.6.40.126:46879
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6848
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.67
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:46:47.897426 6848 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:46:47.897706 6848 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:46:47.904906 6854 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:47.905014 6853 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:47.904968 6857 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:46:47.905429 6848 server_base.cc:1034] running on GCE node
I20260806 22:46:47.905611 6848 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:46:47.905829 6848 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:46:47.907007 6848 hybrid_clock.cc:648] HybridClock initialized: now 1786056407906996 us; error 43 us; skew 500 ppm
Aug 06 22:46:47 dist-test-slave-zh2d krb5kdc[6315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056407, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:46:47.910395 6848 init.cc:377] Logged in from keytab as kudu/127.6.40.67@KRBTEST.COM (short username kudu)
I20260806 22:46:47.910904 6848 webserver.cc:460] Webserver started at http://127.6.40.67:41201/ using document root <none> and password file <none>
I20260806 22:46:47.911175 6848 fs_manager.cc:359] Metadata directory not provided
I20260806 22:46:47.911247 6848 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:46:47.911406 6848 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:46:47.912750 6848 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/minicluster-data/ts-2/data/instance:
uuid: "bad2ec1d93aa4df199bb73bb5facaabb"
format_stamp: "Formatted at 2026-08-06 22:46:47 on dist-test-slave-zh2d"
server_key: "bb00a20cd7fb5a9d95504a63631c5823"
server_key_iv: "676b2fcab6b1fd2eef269b6d7f136830"
server_key_version: "encryptionkey@0"
I20260806 22:46:47.913296 6848 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/minicluster-data/ts-2/wal/instance:
uuid: "bad2ec1d93aa4df199bb73bb5facaabb"
format_stamp: "Formatted at 2026-08-06 22:46:47 on dist-test-slave-zh2d"
server_key: "bb00a20cd7fb5a9d95504a63631c5823"
server_key_iv: "676b2fcab6b1fd2eef269b6d7f136830"
server_key_version: "encryptionkey@0"
I20260806 22:46:47.915062 6848 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.001s
I20260806 22:46:47.916131 6865 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:47.916379 6848 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:46:47.916515 6848 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/minicluster-data/ts-2/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/minicluster-data/ts-2/wal
uuid: "bad2ec1d93aa4df199bb73bb5facaabb"
format_stamp: "Formatted at 2026-08-06 22:46:47 on dist-test-slave-zh2d"
server_key: "bb00a20cd7fb5a9d95504a63631c5823"
server_key_iv: "676b2fcab6b1fd2eef269b6d7f136830"
server_key_version: "encryptionkey@0"
I20260806 22:46:47.916606 6848 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:46:47.929214 6848 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:46:47.931419 6848 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:46:47.931640 6848 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:46:47.932041 6848 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:46:47.932427 6848 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:46:47.932487 6848 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:47.932524 6848 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:46:47.932572 6848 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:47.949954 6848 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.67:35119
I20260806 22:46:47.950152 7030 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.67:35119 every 8 connection(s)
I20260806 22:46:47.950451 6848 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
I20260806 22:46:47.955448 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 6848
I20260806 22:46:47.955554 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056405724609-6305-0/minicluster-data/ts-2/wal/instance
I20260806 22:46:47.955724 6305 external_mini_cluster.cc:1428] Setting key 912a8826fdd170b7bf7a604949367209
Aug 06 22:46:47 dist-test-slave-zh2d krb5kdc[6315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056407, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.67@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
I20260806 22:46:47.962323 6438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:47.951446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.67:46773 (local address 127.6.40.126:46879)
0806 22:46:47.951506 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:47.951508 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:47.951865 (+ 357us) server_negotiation.cc:407] Connection header received
0806 22:46:47.952756 (+ 891us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:47.952757 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:47.952774 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:47.952885 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:47.954241 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:47.954770 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:47.956221 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:47.956433 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:47.956548 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:47.959364 (+ 2816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:47.959366 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:47.959366 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:46:47.959374 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:47.960781 (+ 1407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:47.961282 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:47.961283 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:47.961284 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:47.961350 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:47.961639 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:47.961640 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:47.961641 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:47.961765 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:47.961840 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:47.962140 (+ 300us) server_negotiation.cc:299] Negotiation successful
0806 22:46:47.962227 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 22:46:47.962630 7033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:47.951355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:46879 (local address 127.6.40.67:46773)
0806 22:46:47.951683 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:47.951736 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:47.952613 (+ 877us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:47.952982 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:47.952985 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:47.953757 (+ 772us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:46:47.954125 (+ 368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:47.954131 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:47.954890 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:47.954891 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:47.956076 (+ 1185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:47.956079 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:47.956560 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:47.956562 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:47.956703 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:47.957427 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:46:47.957436 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:46:47.959208 (+ 1772us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:46:47.960952 (+ 1744us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:47.960960 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:47.960962 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:47.961200 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:47.961447 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:47.961448 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:47.961449 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:47.961564 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:47.961843 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:46:47.961851 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:46:47.962001 (+ 150us) client_negotiation.cc:769] Sending connection context
0806 22:46:47.962116 (+ 115us) client_negotiation.cc:240] Negotiation successful
0806 22:46:47.962292 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":165,"threads_started":1}
I20260806 22:46:47.963268 7031 heartbeater.cc:347] Connected to a master server at 127.6.40.126:46879
I20260806 22:46:47.963361 7031 heartbeater.cc:464] Registering TS with master...
I20260806 22:46:47.963517 7031 heartbeater.cc:511] Master 127.6.40.126:46879 requested a full tablet report, sending...
I20260806 22:46:47.963899 6377 ts_manager.cc:194] Registered new tserver with Master: bad2ec1d93aa4df199bb73bb5facaabb (127.6.40.67:35119)
I20260806 22:46:47.964242 6377 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.40.67@KRBTEST.COM'} at 127.6.40.67:46773
I20260806 22:46:47.967636 6305 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 22:46:47 dist-test-slave-zh2d krb5kdc[6315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056407, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 22:46:47 dist-test-slave-zh2d krb5kdc[6315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056407, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.6.40.65@KRBTEST.COM
I20260806 22:46:47.997242 7038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:47.987314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60628 (local address 127.6.40.65:46741)
0806 22:46:47.987493 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:47.987499 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:47.987803 (+ 304us) server_negotiation.cc:407] Connection header received
0806 22:46:47.987840 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:47.987843 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:47.987859 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:47.987948 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:47.988585 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:47.989730 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:47.990602 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:47.990953 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:47.991013 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:47.993724 (+ 2711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:47.993735 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:47.993735 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:46:47.993747 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:47.995853 (+ 2106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:47.996243 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:47.996255 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:47.996255 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:47.996312 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:47.996524 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:47.996525 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:47.996526 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:47.996676 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:47.996857 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:47.996969 (+ 112us) server_negotiation.cc:299] Negotiation successful
0806 22:46:47.997053 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":119,"threads_started":1}
W20260806 22:46:47.997797 6527 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60628
Aug 06 22:46:48 dist-test-slave-zh2d krb5kdc[6315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056408, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 22:46:48 dist-test-slave-zh2d krb5kdc[6315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056408, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.40.65@KRBTEST.COM
I20260806 22:46:48.025595 7038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:48.017757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60644 (local address 127.6.40.65:46741)
0806 22:46:48.017832 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:48.017834 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:48.017930 (+ 96us) server_negotiation.cc:407] Connection header received
0806 22:46:48.018054 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:48.018055 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:48.018075 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:48.018144 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:48.018728 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:48.019278 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:48.020122 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:48.020398 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:48.020552 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:48.022896 (+ 2344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:48.022897 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:48.022897 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:46:48.022904 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:48.024372 (+ 1468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:48.024736 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:48.024738 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:48.024738 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:48.024800 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:48.025057 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:48.025059 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:48.025059 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:48.025168 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:48.025301 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:48.025385 (+ 84us) server_negotiation.cc:299] Negotiation successful
0806 22:46:48.025475 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 22:46:48.026335 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 6466
I20260806 22:46:48.036312 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 6660
I20260806 22:46:48.051419 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 6848
I20260806 22:46:48.059558 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 6336
2026-08-06T22:46:48Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2346 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:46:48 dist-test-slave-zh2d krb5kdc[7050](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:46:48 dist-test-slave-zh2d krb5kdc[7050](info): set up 2 sockets
Aug 06 22:46:48 dist-test-slave-zh2d krb5kdc[7050](info): commencing operation
krb5kdc: starting...
W20260806 22:46:49.342801 6305 mini_kdc.cc:121] Time spent starting KDC: real 1.259s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/krb5kdc/test-user.keytab.
Aug 06 22:46:49 dist-test-slave-zh2d krb5kdc[7050](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056409, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:46:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:46:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.40.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.40.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.40.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab.
Entry for principal kudu/127.6.40.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab.
I20260806 22:46:49.465647 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.126:38685
--webserver_interface=127.6.40.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab
--principal=kudu/127.6.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:36485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.40.126:38685
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:46:49.549554 7069 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:46:49.549749 7069 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:46:49.549772 7069 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:46:49.550853 7069 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:46:49.550886 7069 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:46:49.550901 7069 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:46:49.550913 7069 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:46:49.550926 7069 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:46:49.552315 7069 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:36485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.40.126:38685
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab
--principal=kudu/127.6.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.126:38685
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7069
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:46:49.552513 7069 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:46:49.552682 7069 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:46:49.557610 7079 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:49.557650 7076 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:46:49.557680 7069 server_base.cc:1034] running on GCE node
W20260806 22:46:49.557643 7075 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:46:49.558194 7069 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:46:49.558521 7069 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:46:49.559691 7069 hybrid_clock.cc:648] HybridClock initialized: now 1786056409559684 us; error 38 us; skew 500 ppm
Aug 06 22:46:49 dist-test-slave-zh2d krb5kdc[7050](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056409, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:46:49.562279 7069 init.cc:377] Logged in from keytab as kudu/127.6.40.126@KRBTEST.COM (short username kudu)
I20260806 22:46:49.562773 7069 webserver.cc:460] Webserver started at http://127.6.40.126:32849/ using document root <none> and password file <none>
I20260806 22:46:49.563022 7069 fs_manager.cc:359] Metadata directory not provided
I20260806 22:46:49.563069 7069 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:46:49.563241 7069 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:46:49.564280 7069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/minicluster-data/master-0/data/instance:
uuid: "c692c35881eb4193bea6195383262085"
format_stamp: "Formatted at 2026-08-06 22:46:49 on dist-test-slave-zh2d"
server_key: "97fd242d7bc75bfcfa4e3ce04b2530da"
server_key_iv: "da8c18623b4523e029f5c30463fb35d4"
server_key_version: "encryptionkey@0"
I20260806 22:46:49.564656 7069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/minicluster-data/master-0/wal/instance:
uuid: "c692c35881eb4193bea6195383262085"
format_stamp: "Formatted at 2026-08-06 22:46:49 on dist-test-slave-zh2d"
server_key: "97fd242d7bc75bfcfa4e3ce04b2530da"
server_key_iv: "da8c18623b4523e029f5c30463fb35d4"
server_key_version: "encryptionkey@0"
I20260806 22:46:49.566067 7069 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:46:49.566826 7087 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:49.566987 7069 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:46:49.567065 7069 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/minicluster-data/master-0/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/minicluster-data/master-0/wal
uuid: "c692c35881eb4193bea6195383262085"
format_stamp: "Formatted at 2026-08-06 22:46:49 on dist-test-slave-zh2d"
server_key: "97fd242d7bc75bfcfa4e3ce04b2530da"
server_key_iv: "da8c18623b4523e029f5c30463fb35d4"
server_key_version: "encryptionkey@0"
I20260806 22:46:49.567119 7069 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:46:49.581491 7069 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:46:49.583391 7069 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:46:49.583559 7069 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:46:49.587982 7069 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.126:38685
I20260806 22:46:49.588058 7169 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.126:38685 every 8 connection(s)
I20260806 22:46:49.588400 7069 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
I20260806 22:46:49.589370 7170 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:46:49.590615 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 7069
I20260806 22:46:49.590692 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/minicluster-data/master-0/wal/instance
I20260806 22:46:49.590857 6305 external_mini_cluster.cc:1428] Setting key bdd70e0751ed71d6d06416ca610f1af0
I20260806 22:46:49.592036 7170 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c692c35881eb4193bea6195383262085: Bootstrap starting.
I20260806 22:46:49.592816 7170 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c692c35881eb4193bea6195383262085: Neither blocks nor log segments found. Creating new log.
I20260806 22:46:49.593312 7170 log.cc:824] T 00000000000000000000000000000000 P c692c35881eb4193bea6195383262085: Log is configured to *not* fsync() on all Append() calls
I20260806 22:46:49.594101 7170 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c692c35881eb4193bea6195383262085: No bootstrap required, opened a new log
I20260806 22:46:49.596410 7170 raft_consensus.cc:348] T 00000000000000000000000000000000 P c692c35881eb4193bea6195383262085 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c692c35881eb4193bea6195383262085" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 38685 } }
I20260806 22:46:49.596540 7170 raft_consensus.cc:374] T 00000000000000000000000000000000 P c692c35881eb4193bea6195383262085 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:46:49.596562 7170 raft_consensus.cc:729] T 00000000000000000000000000000000 P c692c35881eb4193bea6195383262085 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c692c35881eb4193bea6195383262085, State: Initialized, Role: FOLLOWER
I20260806 22:46:49.596695 7170 consensus_queue.cc:260] T 00000000000000000000000000000000 P c692c35881eb4193bea6195383262085 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c692c35881eb4193bea6195383262085" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 38685 } }
I20260806 22:46:49.596772 7170 raft_consensus.cc:388] T 00000000000000000000000000000000 P c692c35881eb4193bea6195383262085 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:46:49.596796 7170 raft_consensus.cc:482] T 00000000000000000000000000000000 P c692c35881eb4193bea6195383262085 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:46:49.596884 7170 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c692c35881eb4193bea6195383262085 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:46:49.597417 7170 raft_consensus.cc:504] T 00000000000000000000000000000000 P c692c35881eb4193bea6195383262085 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c692c35881eb4193bea6195383262085" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 38685 } }
I20260806 22:46:49.597506 7170 leader_election.cc:302] T 00000000000000000000000000000000 P c692c35881eb4193bea6195383262085 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c692c35881eb4193bea6195383262085; no voters:
I20260806 22:46:49.597715 7170 leader_election.cc:288] T 00000000000000000000000000000000 P c692c35881eb4193bea6195383262085 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:46:49.597743 7178 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c692c35881eb4193bea6195383262085 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:46:49.597855 7178 raft_consensus.cc:686] T 00000000000000000000000000000000 P c692c35881eb4193bea6195383262085 [term 1 LEADER]: Becoming Leader. State: Replica: c692c35881eb4193bea6195383262085, State: Running, Role: LEADER
I20260806 22:46:49.597951 7178 consensus_queue.cc:237] T 00000000000000000000000000000000 P c692c35881eb4193bea6195383262085 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c692c35881eb4193bea6195383262085" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 38685 } }
I20260806 22:46:49.597985 7170 sys_catalog.cc:564] T 00000000000000000000000000000000 P c692c35881eb4193bea6195383262085 [sys.catalog]: configured and running, proceeding with master startup.
Aug 06 22:46:49 dist-test-slave-zh2d krb5kdc[7050](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056409, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
I20260806 22:46:49.598291 7178 sys_catalog.cc:455] T 00000000000000000000000000000000 P c692c35881eb4193bea6195383262085 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c692c35881eb4193bea6195383262085" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c692c35881eb4193bea6195383262085" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 38685 } } }
I20260806 22:46:49.598377 7178 sys_catalog.cc:458] T 00000000000000000000000000000000 P c692c35881eb4193bea6195383262085 [sys.catalog]: This master's current role is: LEADER
I20260806 22:46:49.598556 7178 sys_catalog.cc:455] T 00000000000000000000000000000000 P c692c35881eb4193bea6195383262085 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c692c35881eb4193bea6195383262085. Latest consensus state: current_term: 1 leader_uuid: "c692c35881eb4193bea6195383262085" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c692c35881eb4193bea6195383262085" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 38685 } } }
I20260806 22:46:49.598701 7178 sys_catalog.cc:458] T 00000000000000000000000000000000 P c692c35881eb4193bea6195383262085 [sys.catalog]: This master's current role is: LEADER
I20260806 22:46:49.598850 7184 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:46:49.599577 7184 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:46:49.601673 7184 catalog_manager.cc:1347] Generated new cluster ID: d7091f8c9a484e919e325f2cd99dd431
I20260806 22:46:49.601728 7184 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:46:49.602790 7176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:49.591391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47856 (local address 127.6.40.126:38685)
0806 22:46:49.591567 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:49.591575 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:49.591600 (+ 25us) server_negotiation.cc:407] Connection header received
0806 22:46:49.592023 (+ 423us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:49.592029 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:49.592074 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:49.592231 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:49.593092 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:49.594453 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:49.595215 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:49.595865 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:49.596002 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:49.598869 (+ 2867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:49.598873 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:49.598873 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:46:49.598884 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:49.601066 (+ 2182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:49.601492 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:49.601495 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:49.601495 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:49.601609 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:49.601826 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:49.601827 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:49.601828 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:49.602128 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:49.602269 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:49.602446 (+ 177us) server_negotiation.cc:299] Negotiation successful
0806 22:46:49.602586 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":88,"threads_started":1}
I20260806 22:46:49.607091 7184 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:46:49.607630 7184 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:46:49.616722 7184 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c692c35881eb4193bea6195383262085: Generated new TSK 0
I20260806 22:46:49.616953 7184 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.40.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.40.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.40.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.65.keytab.
Entry for principal kudu/127.6.40.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.65.keytab.
I20260806 22:46:49.652364 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.65:0
--local_ip_for_outbound_sockets=127.6.40.65
--webserver_interface=127.6.40.65
--webserver_port=0
--tserver_master_addrs=127.6.40.126:38685
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.65.keytab
--principal=kudu/127.6.40.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:36485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:46:49.760037 7206 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:46:49.760244 7206 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:46:49.760269 7206 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:46:49.761540 7206 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:46:49.761584 7206 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:46:49.761611 7206 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.65
I20260806 22:46:49.763103 7206 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:36485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.65.keytab
--principal=kudu/127.6.40.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.40.65
--webserver_port=0
--tserver_master_addrs=127.6.40.126:38685
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7206
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.65
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:46:49.763358 7206 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:46:49.763598 7206 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:46:49.769191 7214 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:46:49.769222 7206 server_base.cc:1034] running on GCE node
W20260806 22:46:49.769192 7213 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:49.769223 7216 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:46:49.769609 7206 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:46:49.769855 7206 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:46:49.771018 7206 hybrid_clock.cc:648] HybridClock initialized: now 1786056409771004 us; error 30 us; skew 500 ppm
Aug 06 22:46:49 dist-test-slave-zh2d krb5kdc[7050](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056409, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:46:49.773573 7206 init.cc:377] Logged in from keytab as kudu/127.6.40.65@KRBTEST.COM (short username kudu)
I20260806 22:46:49.774160 7206 webserver.cc:460] Webserver started at http://127.6.40.65:40535/ using document root <none> and password file <none>
I20260806 22:46:49.774449 7206 fs_manager.cc:359] Metadata directory not provided
I20260806 22:46:49.774521 7206 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:46:49.774674 7206 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:46:49.775833 7206 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/minicluster-data/ts-0/data/instance:
uuid: "649dfaad3f244e1bab2b6148a2eaa807"
format_stamp: "Formatted at 2026-08-06 22:46:49 on dist-test-slave-zh2d"
server_key: "10d4bd8452af24cb2500b2307b0e5ea8"
server_key_iv: "9bf42938bc8d44dd58dc2d72a300a49b"
server_key_version: "encryptionkey@0"
I20260806 22:46:49.776247 7206 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/minicluster-data/ts-0/wal/instance:
uuid: "649dfaad3f244e1bab2b6148a2eaa807"
format_stamp: "Formatted at 2026-08-06 22:46:49 on dist-test-slave-zh2d"
server_key: "10d4bd8452af24cb2500b2307b0e5ea8"
server_key_iv: "9bf42938bc8d44dd58dc2d72a300a49b"
server_key_version: "encryptionkey@0"
I20260806 22:46:49.777814 7206 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:46:49.778664 7224 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:49.778842 7206 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:46:49.778962 7206 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/minicluster-data/ts-0/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/minicluster-data/ts-0/wal
uuid: "649dfaad3f244e1bab2b6148a2eaa807"
format_stamp: "Formatted at 2026-08-06 22:46:49 on dist-test-slave-zh2d"
server_key: "10d4bd8452af24cb2500b2307b0e5ea8"
server_key_iv: "9bf42938bc8d44dd58dc2d72a300a49b"
server_key_version: "encryptionkey@0"
I20260806 22:46:49.779047 7206 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:46:49.789237 7206 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:46:49.791003 7206 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:46:49.791167 7206 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:46:49.791399 7206 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:46:49.791720 7206 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:46:49.791774 7206 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:49.791808 7206 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:46:49.791836 7206 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:49.798451 7206 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.65:35251
I20260806 22:46:49.798507 7381 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.65:35251 every 8 connection(s)
I20260806 22:46:49.798862 7206 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
I20260806 22:46:49.799196 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 7206
I20260806 22:46:49.799307 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/minicluster-data/ts-0/wal/instance
I20260806 22:46:49.799489 6305 external_mini_cluster.cc:1428] Setting key 3afe97ae78850ee10f2a981a51247482
Aug 06 22:46:49 dist-test-slave-zh2d krb5kdc[7050](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056409, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.65@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.40.66@KRBTEST.COM; defaulting to no policy
I20260806 22:46:49.810607 7176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:49.799894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.65:48821 (local address 127.6.40.126:38685)
0806 22:46:49.799946 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:49.799948 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:49.800235 (+ 287us) server_negotiation.cc:407] Connection header received
0806 22:46:49.801849 (+ 1614us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:49.801850 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:49.801862 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:49.801918 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:49.802723 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:49.803555 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:49.804566 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:49.804957 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:49.805039 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:49.807515 (+ 2476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:49.807516 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:49.807517 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:46:49.807527 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:49.809260 (+ 1733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:49.809734 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:49.809735 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:49.809735 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:49.809803 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:49.810014 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:49.810015 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:49.810015 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:49.810109 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:49.810156 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:49.810461 (+ 305us) server_negotiation.cc:299] Negotiation successful
0806 22:46:49.810538 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 22:46:49.810913 7385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:49.799937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:38685 (local address 127.6.40.65:48821)
0806 22:46:49.800107 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:49.800150 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:49.801725 (+ 1575us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:49.802001 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:49.802006 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:49.802417 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:46:49.802622 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:49.802627 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:49.803650 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:49.803651 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:49.804468 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:49.804470 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:49.805048 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:49.805049 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:49.805158 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:49.805760 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:46:49.805769 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:46:49.807416 (+ 1647us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:46:49.809368 (+ 1952us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:49.809376 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:49.809377 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:49.809657 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:49.809868 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:49.809868 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:49.809868 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:49.809972 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:49.810159 (+ 187us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:46:49.810164 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:46:49.810281 (+ 117us) client_negotiation.cc:769] Sending connection context
0806 22:46:49.810389 (+ 108us) client_negotiation.cc:240] Negotiation successful
0806 22:46:49.810663 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":98,"threads_started":1}
I20260806 22:46:49.811775 7383 heartbeater.cc:347] Connected to a master server at 127.6.40.126:38685
I20260806 22:46:49.811896 7383 heartbeater.cc:464] Registering TS with master...
I20260806 22:46:49.812091 7383 heartbeater.cc:511] Master 127.6.40.126:38685 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.40.66@KRBTEST.COM" created.
I20260806 22:46:49.812759 7117 ts_manager.cc:194] Registered new tserver with Master: 649dfaad3f244e1bab2b6148a2eaa807 (127.6.40.65:35251)
I20260806 22:46:49.813463 7117 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.40.65@KRBTEST.COM'} at 127.6.40.65:48821
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.40.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.66.keytab.
Entry for principal kudu/127.6.40.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.66.keytab.
I20260806 22:46:49.828258 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.66:0
--local_ip_for_outbound_sockets=127.6.40.66
--webserver_interface=127.6.40.66
--webserver_port=0
--tserver_master_addrs=127.6.40.126:38685
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.66.keytab
--principal=kudu/127.6.40.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:36485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:46:49.919617 7389 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:46:49.919858 7389 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:46:49.919888 7389 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:46:49.921167 7389 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:46:49.921211 7389 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:46:49.921243 7389 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.66
I20260806 22:46:49.922885 7389 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:36485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.66.keytab
--principal=kudu/127.6.40.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.40.66
--webserver_port=0
--tserver_master_addrs=127.6.40.126:38685
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7389
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.66
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:46:49.923117 7389 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:46:49.923346 7389 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:46:49.929033 7398 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:49.929036 7399 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:46:49.929096 7389 server_base.cc:1034] running on GCE node
W20260806 22:46:49.929045 7401 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:46:49.929469 7389 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:46:49.929733 7389 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:46:49.930982 7389 hybrid_clock.cc:648] HybridClock initialized: now 1786056409930958 us; error 47 us; skew 500 ppm
Aug 06 22:46:49 dist-test-slave-zh2d krb5kdc[7050](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056409, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:46:49.934041 7389 init.cc:377] Logged in from keytab as kudu/127.6.40.66@KRBTEST.COM (short username kudu)
I20260806 22:46:49.934576 7389 webserver.cc:460] Webserver started at http://127.6.40.66:34685/ using document root <none> and password file <none>
I20260806 22:46:49.934845 7389 fs_manager.cc:359] Metadata directory not provided
I20260806 22:46:49.934937 7389 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:46:49.935098 7389 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:46:49.936304 7389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/minicluster-data/ts-1/data/instance:
uuid: "dd1858aba33442828dbabba668046977"
format_stamp: "Formatted at 2026-08-06 22:46:49 on dist-test-slave-zh2d"
server_key: "20d9e878ac64d756c85a8d1a9245d095"
server_key_iv: "f0acc68bc0765693bbd6594c49e11271"
server_key_version: "encryptionkey@0"
I20260806 22:46:49.936726 7389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/minicluster-data/ts-1/wal/instance:
uuid: "dd1858aba33442828dbabba668046977"
format_stamp: "Formatted at 2026-08-06 22:46:49 on dist-test-slave-zh2d"
server_key: "20d9e878ac64d756c85a8d1a9245d095"
server_key_iv: "f0acc68bc0765693bbd6594c49e11271"
server_key_version: "encryptionkey@0"
I20260806 22:46:49.938339 7389 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:46:49.939218 7408 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:49.939510 7389 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:46:49.939623 7389 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/minicluster-data/ts-1/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/minicluster-data/ts-1/wal
uuid: "dd1858aba33442828dbabba668046977"
format_stamp: "Formatted at 2026-08-06 22:46:49 on dist-test-slave-zh2d"
server_key: "20d9e878ac64d756c85a8d1a9245d095"
server_key_iv: "f0acc68bc0765693bbd6594c49e11271"
server_key_version: "encryptionkey@0"
I20260806 22:46:49.939702 7389 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:46:49.962878 7389 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:46:49.965317 7389 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:46:49.965492 7389 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:46:49.965735 7389 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:46:49.966058 7389 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:46:49.966112 7389 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:49.966153 7389 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:46:49.966182 7389 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:49.972693 7389 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.66:34741
I20260806 22:46:49.972738 7554 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.66:34741 every 8 connection(s)
I20260806 22:46:49.973114 7389 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
I20260806 22:46:49.973631 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 7389
I20260806 22:46:49.973734 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/minicluster-data/ts-1/wal/instance
I20260806 22:46:49.973920 6305 external_mini_cluster.cc:1428] Setting key 0af3c252864efd7ce270a730b86ffabf
Aug 06 22:46:49 dist-test-slave-zh2d krb5kdc[7050](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056409, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.66@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.40.67@KRBTEST.COM; defaulting to no policy
I20260806 22:46:49.983529 7176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:49.973881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.66:40327 (local address 127.6.40.126:38685)
0806 22:46:49.974122 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:49.974125 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:49.974244 (+ 119us) server_negotiation.cc:407] Connection header received
0806 22:46:49.975418 (+ 1174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:49.975419 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:49.975431 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:49.975483 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:49.976131 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:49.976643 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:49.977773 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:49.977995 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:49.978083 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:49.980331 (+ 2248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:49.980332 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:49.980333 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:46:49.980342 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:49.982047 (+ 1705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:49.982485 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:49.982486 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:49.982486 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:49.982554 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:49.982826 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:49.982827 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:49.982827 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:49.982986 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:49.983066 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:49.983326 (+ 260us) server_negotiation.cc:299] Negotiation successful
0806 22:46:49.983423 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":209}
I20260806 22:46:49.983677 7557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:49.973897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:38685 (local address 127.6.40.66:40327)
0806 22:46:49.974106 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:49.974158 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:49.975296 (+ 1138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:49.975567 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:49.975571 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:49.975835 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:46:49.976034 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:49.976038 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:49.976749 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:49.976750 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:49.977680 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:49.977683 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:49.978095 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:49.978096 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:49.978242 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:49.978787 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:46:49.978796 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:46:49.980234 (+ 1438us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:46:49.982131 (+ 1897us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:49.982138 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:49.982140 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:49.982387 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:49.982629 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:49.982629 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:49.982630 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:49.982752 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:49.983061 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:46:49.983067 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:46:49.983183 (+ 116us) client_negotiation.cc:769] Sending connection context
0806 22:46:49.983268 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 22:46:49.983466 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":121,"threads_started":1}
I20260806 22:46:49.984191 7555 heartbeater.cc:347] Connected to a master server at 127.6.40.126:38685
I20260806 22:46:49.984299 7555 heartbeater.cc:464] Registering TS with master...
I20260806 22:46:49.984522 7555 heartbeater.cc:511] Master 127.6.40.126:38685 requested a full tablet report, sending...
I20260806 22:46:49.984972 7117 ts_manager.cc:194] Registered new tserver with Master: dd1858aba33442828dbabba668046977 (127.6.40.66:34741)
I20260806 22:46:49.985307 7117 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.40.66@KRBTEST.COM'} at 127.6.40.66:40327
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.40.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.40.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.67.keytab.
Entry for principal kudu/127.6.40.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.67.keytab.
I20260806 22:46:50.003974 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.67:0
--local_ip_for_outbound_sockets=127.6.40.67
--webserver_interface=127.6.40.67
--webserver_port=0
--tserver_master_addrs=127.6.40.126:38685
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.67.keytab
--principal=kudu/127.6.40.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:36485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:46:50.133776 7561 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:46:50.134003 7561 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:46:50.134032 7561 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:46:50.135311 7561 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:46:50.135352 7561 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:46:50.135414 7561 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.67
I20260806 22:46:50.137148 7561 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:36485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.67.keytab
--principal=kudu/127.6.40.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.40.67
--webserver_port=0
--tserver_master_addrs=127.6.40.126:38685
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7561
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.67
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:46:50.137405 7561 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:46:50.137671 7561 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:46:50.143464 7575 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:50.143518 7569 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:50.143566 7572 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:46:50.143877 7561 server_base.cc:1034] running on GCE node
I20260806 22:46:50.144140 7561 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:46:50.144435 7561 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:46:50.145622 7561 hybrid_clock.cc:648] HybridClock initialized: now 1786056410145598 us; error 38 us; skew 500 ppm
Aug 06 22:46:50 dist-test-slave-zh2d krb5kdc[7050](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056410, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:46:50.148973 7561 init.cc:377] Logged in from keytab as kudu/127.6.40.67@KRBTEST.COM (short username kudu)
I20260806 22:46:50.149420 7561 webserver.cc:460] Webserver started at http://127.6.40.67:45595/ using document root <none> and password file <none>
I20260806 22:46:50.149672 7561 fs_manager.cc:359] Metadata directory not provided
I20260806 22:46:50.149717 7561 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:46:50.149870 7561 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:46:50.151125 7561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/minicluster-data/ts-2/data/instance:
uuid: "f5d69de42cbc4a3aadbcde82a06f67cf"
format_stamp: "Formatted at 2026-08-06 22:46:50 on dist-test-slave-zh2d"
server_key: "5bd0870a84408ad691f8255485ba9bcf"
server_key_iv: "715077524dd45954cf423a691cc21a07"
server_key_version: "encryptionkey@0"
I20260806 22:46:50.151533 7561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/minicluster-data/ts-2/wal/instance:
uuid: "f5d69de42cbc4a3aadbcde82a06f67cf"
format_stamp: "Formatted at 2026-08-06 22:46:50 on dist-test-slave-zh2d"
server_key: "5bd0870a84408ad691f8255485ba9bcf"
server_key_iv: "715077524dd45954cf423a691cc21a07"
server_key_version: "encryptionkey@0"
I20260806 22:46:50.153034 7561 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:46:50.153913 7586 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:50.154098 7561 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:46:50.154206 7561 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/minicluster-data/ts-2/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/minicluster-data/ts-2/wal
uuid: "f5d69de42cbc4a3aadbcde82a06f67cf"
format_stamp: "Formatted at 2026-08-06 22:46:50 on dist-test-slave-zh2d"
server_key: "5bd0870a84408ad691f8255485ba9bcf"
server_key_iv: "715077524dd45954cf423a691cc21a07"
server_key_version: "encryptionkey@0"
I20260806 22:46:50.154276 7561 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:46:50.170621 7561 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:46:50.173310 7561 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:46:50.173527 7561 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:46:50.173972 7561 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:46:50.174341 7561 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:46:50.174398 7561 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:50.174434 7561 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:46:50.174480 7561 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:50.191098 7561 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.67:44655
I20260806 22:46:50.191298 7731 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.67:44655 every 8 connection(s)
I20260806 22:46:50.191779 7561 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
I20260806 22:46:50.200470 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 7561
I20260806 22:46:50.200596 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056405724609-6305-0/minicluster-data/ts-2/wal/instance
I20260806 22:46:50.200752 6305 external_mini_cluster.cc:1428] Setting key 71faad20ae6aa0fcbbd20f7eaf90b1e5
Aug 06 22:46:50 dist-test-slave-zh2d krb5kdc[7050](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056410, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.67@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
I20260806 22:46:50.204885 7176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:50.192974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.67:41753 (local address 127.6.40.126:38685)
0806 22:46:50.193058 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:50.193060 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:50.193296 (+ 236us) server_negotiation.cc:407] Connection header received
0806 22:46:50.194811 (+ 1515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:50.194813 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:50.194827 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:50.194913 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:50.195890 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:50.196746 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:50.198287 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:50.198510 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:50.198603 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:50.201515 (+ 2912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:50.201517 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:50.201517 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:46:50.201526 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:50.202993 (+ 1467us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:50.203604 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:50.203606 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:50.203606 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:50.203688 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:50.204031 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:50.204032 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:50.204032 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:50.204180 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:50.204277 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:50.204614 (+ 337us) server_negotiation.cc:299] Negotiation successful
0806 22:46:50.204718 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 22:46:50.205133 7734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:50.192965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:38685 (local address 127.6.40.67:41753)
0806 22:46:50.193160 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:50.193204 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:50.194679 (+ 1475us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:50.195037 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:50.195042 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:50.195509 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:46:50.195789 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:50.195794 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:50.196875 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:50.196877 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:50.198207 (+ 1330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:50.198211 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:50.198616 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:50.198617 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:50.198860 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:50.199526 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:46:50.199534 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:46:50.201407 (+ 1873us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:46:50.203123 (+ 1716us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:50.203132 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:50.203135 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:50.203493 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:50.203791 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:50.203792 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:50.203793 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:50.203938 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:50.204264 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:46:50.204272 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:46:50.204441 (+ 169us) client_negotiation.cc:769] Sending connection context
0806 22:46:50.204572 (+ 131us) client_negotiation.cc:240] Negotiation successful
0806 22:46:50.204757 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":119,"threads_started":1}
I20260806 22:46:50.205662 7732 heartbeater.cc:347] Connected to a master server at 127.6.40.126:38685
I20260806 22:46:50.205773 7732 heartbeater.cc:464] Registering TS with master...
I20260806 22:46:50.205963 7732 heartbeater.cc:511] Master 127.6.40.126:38685 requested a full tablet report, sending...
I20260806 22:46:50.206396 7117 ts_manager.cc:194] Registered new tserver with Master: f5d69de42cbc4a3aadbcde82a06f67cf (127.6.40.67:44655)
I20260806 22:46:50.206751 7117 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.40.67@KRBTEST.COM'} at 127.6.40.67:41753
I20260806 22:46:50.212450 6305 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:46:50.220290 7176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:50.214234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47870 (local address 127.6.40.126:38685)
0806 22:46:50.214303 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:50.214304 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:50.214502 (+ 198us) server_negotiation.cc:407] Connection header received
0806 22:46:50.214582 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:50.214583 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:50.214593 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:50.214651 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:50.215231 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:50.215718 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:50.216448 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:50.216663 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:50.216740 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:50.217929 (+ 1189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:50.217931 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:50.217931 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:46:50.217938 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:50.219083 (+ 1145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:50.219431 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:50.219432 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:50.219432 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:50.219514 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:50.219731 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:50.219731 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:50.219732 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:50.219886 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:50.220003 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:50.220107 (+ 104us) server_negotiation.cc:299] Negotiation successful
0806 22:46:50.220189 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 22:46:50.222451 7117 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47870:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:46:50.224131 7117 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:46:50.231056 7753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:50.226904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52722 (local address 127.6.40.66:34741)
0806 22:46:50.227137 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:50.227142 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:50.227348 (+ 206us) server_negotiation.cc:407] Connection header received
0806 22:46:50.227415 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:50.227417 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:50.227435 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:50.227508 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:46:50.227739 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:50.228738 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:50.229848 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:50.230599 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:50.230685 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:50.230753 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:50.230922 (+ 169us) server_negotiation.cc:299] Negotiation successful
0806 22:46:50.230972 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":98,"threads_started":1}
I20260806 22:46:50.231067 7744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:50.226579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.67:44655 (local address 127.0.0.1:35088)
0806 22:46:50.226740 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:50.226781 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:50.226850 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:50.227125 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:50.227128 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:50.227130 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:46:50.227201 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:50.227203 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:50.228578 (+ 1375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:50.228578 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:50.229890 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:50.229893 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:50.230749 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:50.230750 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:50.230801 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:50.230815 (+ 14us) client_negotiation.cc:769] Sending connection context
0806 22:46:50.230896 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 22:46:50.230955 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"mutex_wait_us":49,"thread_start_us":58,"threads_started":1}
I20260806 22:46:50.231323 7749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:50.226673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35088 (local address 127.6.40.67:44655)
0806 22:46:50.226944 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:50.226950 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:50.226962 (+ 12us) server_negotiation.cc:407] Connection header received
0806 22:46:50.227004 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:50.227006 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:50.227022 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:50.227114 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:46:50.227300 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:50.228480 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:50.229982 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:50.230672 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:50.230736 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:50.230784 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:50.231195 (+ 411us) server_negotiation.cc:299] Negotiation successful
0806 22:46:50.231241 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":215,"thread_start_us":94,"threads_started":1}
I20260806 22:46:50.231067 7748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:50.226648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.66:34741 (local address 127.0.0.1:52722)
0806 22:46:50.227207 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:50.227216 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:50.227285 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:50.227612 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:50.227613 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:50.227615 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:46:50.227658 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:50.227660 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:50.228860 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:50.228861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:50.229762 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:50.229764 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:50.230692 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:50.230693 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:50.230798 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:50.230815 (+ 17us) client_negotiation.cc:769] Sending connection context
0806 22:46:50.230895 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 22:46:50.230952 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":540,"thread_start_us":70,"threads_started":1}
I20260806 22:46:50.232239 7642 tablet_service.cc:1467] Processing CreateTablet for tablet 6272d6e3c53d408e8e9b8cbb08e5684a (DEFAULT_TABLE table=test-table [id=b72577fee90041a789117b11879ca942]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:46:50.232443 7467 tablet_service.cc:1467] Processing CreateTablet for tablet 6272d6e3c53d408e8e9b8cbb08e5684a (DEFAULT_TABLE table=test-table [id=b72577fee90041a789117b11879ca942]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:46:50.232623 7642 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6272d6e3c53d408e8e9b8cbb08e5684a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:46:50.232802 7467 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6272d6e3c53d408e8e9b8cbb08e5684a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:46:50.233177 7746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:50.226598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.65:35251 (local address 127.0.0.1:58230)
0806 22:46:50.226960 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:50.226985 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:50.227024 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:50.227424 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:50.227425 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:50.227428 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:46:50.227544 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:50.227547 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:50.228502 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:50.228503 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:50.229894 (+ 1391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:50.229896 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:50.231640 (+ 1744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:50.231641 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:50.231706 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:50.231711 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:46:50.233060 (+ 1349us) client_negotiation.cc:240] Negotiation successful
0806 22:46:50.233113 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"mutex_wait_us":38,"thread_start_us":108,"threads_started":1}
I20260806 22:46:50.233502 7751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:50.226886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58230 (local address 127.6.40.65:35251)
0806 22:46:50.227240 (+ 354us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:50.227245 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:50.227256 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:46:50.227298 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:50.227300 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:50.227316 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:50.227416 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:46:50.227631 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:50.228389 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:50.230830 (+ 2441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:50.231556 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:50.231616 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:50.231653 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:50.233368 (+ 1715us) server_negotiation.cc:299] Negotiation successful
0806 22:46:50.233423 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":92,"threads_started":1}
I20260806 22:46:50.234160 7295 tablet_service.cc:1467] Processing CreateTablet for tablet 6272d6e3c53d408e8e9b8cbb08e5684a (DEFAULT_TABLE table=test-table [id=b72577fee90041a789117b11879ca942]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:46:50.234498 7295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6272d6e3c53d408e8e9b8cbb08e5684a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:46:50.236321 7756 tablet_bootstrap.cc:492] T 6272d6e3c53d408e8e9b8cbb08e5684a P dd1858aba33442828dbabba668046977: Bootstrap starting.
I20260806 22:46:50.236656 7757 tablet_bootstrap.cc:492] T 6272d6e3c53d408e8e9b8cbb08e5684a P f5d69de42cbc4a3aadbcde82a06f67cf: Bootstrap starting.
I20260806 22:46:50.237402 7757 tablet_bootstrap.cc:654] T 6272d6e3c53d408e8e9b8cbb08e5684a P f5d69de42cbc4a3aadbcde82a06f67cf: Neither blocks nor log segments found. Creating new log.
I20260806 22:46:50.237743 7758 tablet_bootstrap.cc:492] T 6272d6e3c53d408e8e9b8cbb08e5684a P 649dfaad3f244e1bab2b6148a2eaa807: Bootstrap starting.
I20260806 22:46:50.238045 7757 log.cc:824] T 6272d6e3c53d408e8e9b8cbb08e5684a P f5d69de42cbc4a3aadbcde82a06f67cf: Log is configured to *not* fsync() on all Append() calls
I20260806 22:46:50.238430 7758 tablet_bootstrap.cc:654] T 6272d6e3c53d408e8e9b8cbb08e5684a P 649dfaad3f244e1bab2b6148a2eaa807: Neither blocks nor log segments found. Creating new log.
I20260806 22:46:50.238864 7756 tablet_bootstrap.cc:654] T 6272d6e3c53d408e8e9b8cbb08e5684a P dd1858aba33442828dbabba668046977: Neither blocks nor log segments found. Creating new log.
I20260806 22:46:50.238929 7758 log.cc:824] T 6272d6e3c53d408e8e9b8cbb08e5684a P 649dfaad3f244e1bab2b6148a2eaa807: Log is configured to *not* fsync() on all Append() calls
I20260806 22:46:50.239348 7756 log.cc:824] T 6272d6e3c53d408e8e9b8cbb08e5684a P dd1858aba33442828dbabba668046977: Log is configured to *not* fsync() on all Append() calls
I20260806 22:46:50.239691 7758 tablet_bootstrap.cc:492] T 6272d6e3c53d408e8e9b8cbb08e5684a P 649dfaad3f244e1bab2b6148a2eaa807: No bootstrap required, opened a new log
I20260806 22:46:50.239773 7758 ts_tablet_manager.cc:1397] T 6272d6e3c53d408e8e9b8cbb08e5684a P 649dfaad3f244e1bab2b6148a2eaa807: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:46:50.239892 7757 tablet_bootstrap.cc:492] T 6272d6e3c53d408e8e9b8cbb08e5684a P f5d69de42cbc4a3aadbcde82a06f67cf: No bootstrap required, opened a new log
I20260806 22:46:50.239975 7757 ts_tablet_manager.cc:1397] T 6272d6e3c53d408e8e9b8cbb08e5684a P f5d69de42cbc4a3aadbcde82a06f67cf: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 22:46:50.241268 7758 raft_consensus.cc:348] T 6272d6e3c53d408e8e9b8cbb08e5684a P 649dfaad3f244e1bab2b6148a2eaa807 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "649dfaad3f244e1bab2b6148a2eaa807" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 35251 } } peers { permanent_uuid: "f5d69de42cbc4a3aadbcde82a06f67cf" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 44655 } } peers { permanent_uuid: "dd1858aba33442828dbabba668046977" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 34741 } }
I20260806 22:46:50.241410 7758 raft_consensus.cc:374] T 6272d6e3c53d408e8e9b8cbb08e5684a P 649dfaad3f244e1bab2b6148a2eaa807 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:46:50.241433 7758 raft_consensus.cc:729] T 6272d6e3c53d408e8e9b8cbb08e5684a P 649dfaad3f244e1bab2b6148a2eaa807 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 649dfaad3f244e1bab2b6148a2eaa807, State: Initialized, Role: FOLLOWER
I20260806 22:46:50.241379 7757 raft_consensus.cc:348] T 6272d6e3c53d408e8e9b8cbb08e5684a P f5d69de42cbc4a3aadbcde82a06f67cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "649dfaad3f244e1bab2b6148a2eaa807" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 35251 } } peers { permanent_uuid: "f5d69de42cbc4a3aadbcde82a06f67cf" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 44655 } } peers { permanent_uuid: "dd1858aba33442828dbabba668046977" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 34741 } }
I20260806 22:46:50.241588 7757 raft_consensus.cc:374] T 6272d6e3c53d408e8e9b8cbb08e5684a P f5d69de42cbc4a3aadbcde82a06f67cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:46:50.241603 7758 consensus_queue.cc:260] T 6272d6e3c53d408e8e9b8cbb08e5684a P 649dfaad3f244e1bab2b6148a2eaa807 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "649dfaad3f244e1bab2b6148a2eaa807" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 35251 } } peers { permanent_uuid: "f5d69de42cbc4a3aadbcde82a06f67cf" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 44655 } } peers { permanent_uuid: "dd1858aba33442828dbabba668046977" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 34741 } }
I20260806 22:46:50.241658 7757 raft_consensus.cc:729] T 6272d6e3c53d408e8e9b8cbb08e5684a P f5d69de42cbc4a3aadbcde82a06f67cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5d69de42cbc4a3aadbcde82a06f67cf, State: Initialized, Role: FOLLOWER
I20260806 22:46:50.241797 7757 consensus_queue.cc:260] T 6272d6e3c53d408e8e9b8cbb08e5684a P f5d69de42cbc4a3aadbcde82a06f67cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "649dfaad3f244e1bab2b6148a2eaa807" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 35251 } } peers { permanent_uuid: "f5d69de42cbc4a3aadbcde82a06f67cf" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 44655 } } peers { permanent_uuid: "dd1858aba33442828dbabba668046977" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 34741 } }
I20260806 22:46:50.241900 7758 ts_tablet_manager.cc:1428] T 6272d6e3c53d408e8e9b8cbb08e5684a P 649dfaad3f244e1bab2b6148a2eaa807: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260806 22:46:50.241947 7383 heartbeater.cc:503] Master 127.6.40.126:38685 was elected leader, sending a full tablet report...
I20260806 22:46:50.242062 7757 ts_tablet_manager.cc:1428] T 6272d6e3c53d408e8e9b8cbb08e5684a P f5d69de42cbc4a3aadbcde82a06f67cf: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:46:50.242101 7732 heartbeater.cc:503] Master 127.6.40.126:38685 was elected leader, sending a full tablet report...
I20260806 22:46:50.242496 7756 tablet_bootstrap.cc:492] T 6272d6e3c53d408e8e9b8cbb08e5684a P dd1858aba33442828dbabba668046977: No bootstrap required, opened a new log
I20260806 22:46:50.242581 7756 ts_tablet_manager.cc:1397] T 6272d6e3c53d408e8e9b8cbb08e5684a P dd1858aba33442828dbabba668046977: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260806 22:46:50.244222 7756 raft_consensus.cc:348] T 6272d6e3c53d408e8e9b8cbb08e5684a P dd1858aba33442828dbabba668046977 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "649dfaad3f244e1bab2b6148a2eaa807" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 35251 } } peers { permanent_uuid: "f5d69de42cbc4a3aadbcde82a06f67cf" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 44655 } } peers { permanent_uuid: "dd1858aba33442828dbabba668046977" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 34741 } }
I20260806 22:46:50.244371 7756 raft_consensus.cc:374] T 6272d6e3c53d408e8e9b8cbb08e5684a P dd1858aba33442828dbabba668046977 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:46:50.244418 7756 raft_consensus.cc:729] T 6272d6e3c53d408e8e9b8cbb08e5684a P dd1858aba33442828dbabba668046977 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd1858aba33442828dbabba668046977, State: Initialized, Role: FOLLOWER
I20260806 22:46:50.244536 7756 consensus_queue.cc:260] T 6272d6e3c53d408e8e9b8cbb08e5684a P dd1858aba33442828dbabba668046977 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "649dfaad3f244e1bab2b6148a2eaa807" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 35251 } } peers { permanent_uuid: "f5d69de42cbc4a3aadbcde82a06f67cf" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 44655 } } peers { permanent_uuid: "dd1858aba33442828dbabba668046977" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 34741 } }
I20260806 22:46:50.244799 7756 ts_tablet_manager.cc:1428] T 6272d6e3c53d408e8e9b8cbb08e5684a P dd1858aba33442828dbabba668046977: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:46:50.244884 7555 heartbeater.cc:503] Master 127.6.40.126:38685 was elected leader, sending a full tablet report...
I20260806 22:46:50.287372 7766 raft_consensus.cc:482] T 6272d6e3c53d408e8e9b8cbb08e5684a P f5d69de42cbc4a3aadbcde82a06f67cf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:46:50.287508 7766 raft_consensus.cc:504] T 6272d6e3c53d408e8e9b8cbb08e5684a P f5d69de42cbc4a3aadbcde82a06f67cf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "649dfaad3f244e1bab2b6148a2eaa807" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 35251 } } peers { permanent_uuid: "f5d69de42cbc4a3aadbcde82a06f67cf" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 44655 } } peers { permanent_uuid: "dd1858aba33442828dbabba668046977" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 34741 } }
I20260806 22:46:50.288128 7766 leader_election.cc:288] T 6272d6e3c53d408e8e9b8cbb08e5684a P f5d69de42cbc4a3aadbcde82a06f67cf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 649dfaad3f244e1bab2b6148a2eaa807 (127.6.40.65:35251), dd1858aba33442828dbabba668046977 (127.6.40.66:34741)
I20260806 22:46:50.292091 7753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:50.288432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.67:35903 (local address 127.6.40.66:34741)
0806 22:46:50.288478 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:50.288480 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:50.288604 (+ 124us) server_negotiation.cc:407] Connection header received
0806 22:46:50.288664 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:50.288665 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:50.288686 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:50.288752 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:46:50.289075 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:50.289758 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:50.291064 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:50.291631 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:50.291713 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:50.291725 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:50.291967 (+ 242us) server_negotiation.cc:299] Negotiation successful
0806 22:46:50.292018 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 22:46:50.292091 7751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:50.288394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.67:48401 (local address 127.6.40.65:35251)
0806 22:46:50.288457 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:50.288460 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:50.288470 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:46:50.288503 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:50.288504 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:50.288523 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:50.288601 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:46:50.288768 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:50.289602 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:50.291064 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:50.291651 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:50.291713 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:50.291724 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:50.291967 (+ 243us) server_negotiation.cc:299] Negotiation successful
0806 22:46:50.292015 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 22:46:50.292085 7734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:50.288278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.65:35251 (local address 127.6.40.67:48401)
0806 22:46:50.288331 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:50.288346 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:50.288396 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:50.288609 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:50.288610 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:50.288613 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:46:50.288692 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:50.288694 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:50.289681 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:50.289681 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:50.290953 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:50.290956 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:50.291731 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:50.291732 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:50.291857 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:50.291859 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 22:46:50.291916 (+ 57us) client_negotiation.cc:240] Negotiation successful
0806 22:46:50.292006 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26,"mutex_wait_us":53}
I20260806 22:46:50.292097 7769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:50.288327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.66:34741 (local address 127.6.40.67:35903)
0806 22:46:50.288526 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:50.288534 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:50.288602 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:50.288756 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:50.288757 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:50.288759 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:46:50.288966 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:50.288983 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:50.289833 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:50.289834 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:50.290963 (+ 1129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:50.290965 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:50.291731 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:50.291732 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:50.291846 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:50.291851 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:46:50.291916 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 22:46:50.291994 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"mutex_wait_us":16,"thread_start_us":130,"threads_started":1}
I20260806 22:46:50.292541 7491 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6272d6e3c53d408e8e9b8cbb08e5684a" candidate_uuid: "f5d69de42cbc4a3aadbcde82a06f67cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd1858aba33442828dbabba668046977" is_pre_election: true
I20260806 22:46:50.292791 7491 raft_consensus.cc:2454] T 6272d6e3c53d408e8e9b8cbb08e5684a P dd1858aba33442828dbabba668046977 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5d69de42cbc4a3aadbcde82a06f67cf in term 0.
I20260806 22:46:50.292721 7320 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6272d6e3c53d408e8e9b8cbb08e5684a" candidate_uuid: "f5d69de42cbc4a3aadbcde82a06f67cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "649dfaad3f244e1bab2b6148a2eaa807" is_pre_election: true
I20260806 22:46:50.292960 7320 raft_consensus.cc:2454] T 6272d6e3c53d408e8e9b8cbb08e5684a P 649dfaad3f244e1bab2b6148a2eaa807 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5d69de42cbc4a3aadbcde82a06f67cf in term 0.
I20260806 22:46:50.293088 7588 leader_election.cc:302] T 6272d6e3c53d408e8e9b8cbb08e5684a P f5d69de42cbc4a3aadbcde82a06f67cf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dd1858aba33442828dbabba668046977, f5d69de42cbc4a3aadbcde82a06f67cf; no voters:
I20260806 22:46:50.293262 7766 raft_consensus.cc:2781] T 6272d6e3c53d408e8e9b8cbb08e5684a P f5d69de42cbc4a3aadbcde82a06f67cf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:46:50.293349 7766 raft_consensus.cc:482] T 6272d6e3c53d408e8e9b8cbb08e5684a P f5d69de42cbc4a3aadbcde82a06f67cf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:46:50.293385 7766 raft_consensus.cc:3037] T 6272d6e3c53d408e8e9b8cbb08e5684a P f5d69de42cbc4a3aadbcde82a06f67cf [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:46:50.294183 7766 raft_consensus.cc:504] T 6272d6e3c53d408e8e9b8cbb08e5684a P f5d69de42cbc4a3aadbcde82a06f67cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "649dfaad3f244e1bab2b6148a2eaa807" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 35251 } } peers { permanent_uuid: "f5d69de42cbc4a3aadbcde82a06f67cf" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 44655 } } peers { permanent_uuid: "dd1858aba33442828dbabba668046977" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 34741 } }
I20260806 22:46:50.294340 7766 leader_election.cc:288] T 6272d6e3c53d408e8e9b8cbb08e5684a P f5d69de42cbc4a3aadbcde82a06f67cf [CANDIDATE]: Term 1 election: Requested vote from peers 649dfaad3f244e1bab2b6148a2eaa807 (127.6.40.65:35251), dd1858aba33442828dbabba668046977 (127.6.40.66:34741)
I20260806 22:46:50.294489 7320 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6272d6e3c53d408e8e9b8cbb08e5684a" candidate_uuid: "f5d69de42cbc4a3aadbcde82a06f67cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "649dfaad3f244e1bab2b6148a2eaa807"
I20260806 22:46:50.294498 7491 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6272d6e3c53d408e8e9b8cbb08e5684a" candidate_uuid: "f5d69de42cbc4a3aadbcde82a06f67cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd1858aba33442828dbabba668046977"
I20260806 22:46:50.294570 7491 raft_consensus.cc:3037] T 6272d6e3c53d408e8e9b8cbb08e5684a P dd1858aba33442828dbabba668046977 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:46:50.294586 7320 raft_consensus.cc:3037] T 6272d6e3c53d408e8e9b8cbb08e5684a P 649dfaad3f244e1bab2b6148a2eaa807 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:46:50.295320 7320 raft_consensus.cc:2454] T 6272d6e3c53d408e8e9b8cbb08e5684a P 649dfaad3f244e1bab2b6148a2eaa807 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5d69de42cbc4a3aadbcde82a06f67cf in term 1.
I20260806 22:46:50.295356 7491 raft_consensus.cc:2454] T 6272d6e3c53d408e8e9b8cbb08e5684a P dd1858aba33442828dbabba668046977 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5d69de42cbc4a3aadbcde82a06f67cf in term 1.
I20260806 22:46:50.295513 7588 leader_election.cc:302] T 6272d6e3c53d408e8e9b8cbb08e5684a P f5d69de42cbc4a3aadbcde82a06f67cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dd1858aba33442828dbabba668046977, f5d69de42cbc4a3aadbcde82a06f67cf; no voters:
I20260806 22:46:50.295612 7766 raft_consensus.cc:2781] T 6272d6e3c53d408e8e9b8cbb08e5684a P f5d69de42cbc4a3aadbcde82a06f67cf [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:46:50.295791 7766 raft_consensus.cc:686] T 6272d6e3c53d408e8e9b8cbb08e5684a P f5d69de42cbc4a3aadbcde82a06f67cf [term 1 LEADER]: Becoming Leader. State: Replica: f5d69de42cbc4a3aadbcde82a06f67cf, State: Running, Role: LEADER
I20260806 22:46:50.295895 7766 consensus_queue.cc:237] T 6272d6e3c53d408e8e9b8cbb08e5684a P f5d69de42cbc4a3aadbcde82a06f67cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "649dfaad3f244e1bab2b6148a2eaa807" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 35251 } } peers { permanent_uuid: "f5d69de42cbc4a3aadbcde82a06f67cf" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 44655 } } peers { permanent_uuid: "dd1858aba33442828dbabba668046977" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 34741 } }
I20260806 22:46:50.296805 7116 catalog_manager.cc:5495] T 6272d6e3c53d408e8e9b8cbb08e5684a P f5d69de42cbc4a3aadbcde82a06f67cf reported cstate change: term changed from 0 to 1, leader changed from <none> to f5d69de42cbc4a3aadbcde82a06f67cf (127.6.40.67). New cstate: current_term: 1 leader_uuid: "f5d69de42cbc4a3aadbcde82a06f67cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "649dfaad3f244e1bab2b6148a2eaa807" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 35251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5d69de42cbc4a3aadbcde82a06f67cf" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 44655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1858aba33442828dbabba668046977" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 34741 } health_report { overall_health: UNKNOWN } } }
W20260806 22:46:50.299770 7384 tablet.cc:2369] T 6272d6e3c53d408e8e9b8cbb08e5684a P 649dfaad3f244e1bab2b6148a2eaa807: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 06 22:46:50 dist-test-slave-zh2d krb5kdc[7050](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056409, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.40.65@KRBTEST.COM
I20260806 22:46:50.307171 7751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:50.299801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58236 (local address 127.6.40.65:35251)
0806 22:46:50.299840 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:50.299842 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:50.300054 (+ 212us) server_negotiation.cc:407] Connection header received
0806 22:46:50.300121 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:50.300122 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:50.300133 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:50.300183 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:50.300809 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:50.301311 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:50.302043 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:50.302240 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:50.302326 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:50.304677 (+ 2351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:50.304686 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:50.304687 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:46:50.304693 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:50.306158 (+ 1465us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:50.306501 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:50.306504 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:50.306504 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:50.306558 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:50.306723 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:50.306723 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:50.306723 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:50.306845 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:50.306958 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:50.307006 (+ 48us) server_negotiation.cc:299] Negotiation successful
0806 22:46:50.307066 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
Aug 06 22:46:50 dist-test-slave-zh2d krb5kdc[7050](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056410, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 22:46:50 dist-test-slave-zh2d krb5kdc[7050](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056410, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.6.40.65@KRBTEST.COM
I20260806 22:46:50.331964 7751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:50.324955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58242 (local address 127.6.40.65:35251)
0806 22:46:50.325035 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:50.325037 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:50.325137 (+ 100us) server_negotiation.cc:407] Connection header received
0806 22:46:50.325217 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:50.325218 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:50.325234 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:50.325290 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:50.325850 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:50.326320 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:50.327058 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:50.327254 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:50.327330 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:50.329521 (+ 2191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:50.329522 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:50.329523 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:46:50.329530 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:50.330770 (+ 1240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:50.331121 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:50.331122 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:50.331123 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:50.331182 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:50.331382 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:50.331383 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:50.331384 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:50.331490 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:50.331597 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:50.331776 (+ 179us) server_negotiation.cc:299] Negotiation successful
0806 22:46:50.331854 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
Aug 06 22:46:50 dist-test-slave-zh2d krb5kdc[7050](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056410, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 22:46:50 dist-test-slave-zh2d krb5kdc[7050](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056410, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.40.65@KRBTEST.COM
I20260806 22:46:50.354733 7751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:50.347860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58244 (local address 127.6.40.65:35251)
0806 22:46:50.347975 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:50.347977 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:50.347991 (+ 14us) server_negotiation.cc:407] Connection header received
0806 22:46:50.348067 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:50.348069 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:50.348078 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:50.348128 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:50.348654 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:50.349194 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:50.349952 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:50.350147 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:50.350196 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:50.352291 (+ 2095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:50.352292 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:50.352292 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:46:50.352299 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:50.353783 (+ 1484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:50.354078 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:50.354079 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:50.354080 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:50.354129 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:50.354288 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:50.354289 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:50.354289 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:50.354372 (+ 83us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:50.354448 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:50.354542 (+ 94us) server_negotiation.cc:299] Negotiation successful
0806 22:46:50.354595 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
W20260806 22:46:50.355353 7268 tablet_service.cc:2908] Rejecting scan request for tablet 6272d6e3c53d408e8e9b8cbb08e5684a: Uninitialized: clean time has not yet been initialized
I20260806 22:46:50.356231 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 7206
I20260806 22:46:50.371030 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 7389
I20260806 22:46:50.381273 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 7561
I20260806 22:46:50.390508 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 7069
2026-08-06T22:46:50Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2332 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:46:50 dist-test-slave-zh2d krb5kdc[7792](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:46:50 dist-test-slave-zh2d krb5kdc[7792](info): set up 2 sockets
Aug 06 22:46:50 dist-test-slave-zh2d krb5kdc[7792](info): commencing operation
krb5kdc: starting...
W20260806 22:46:51.593173 6305 mini_kdc.cc:121] Time spent starting KDC: real 1.176s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/krb5kdc/test-user.keytab.
Aug 06 22:46:51 dist-test-slave-zh2d krb5kdc[7792](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056411, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:46:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:46:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.40.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.40.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.40.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab.
Entry for principal kudu/127.6.40.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab.
I20260806 22:46:51.720297 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.126:44775
--webserver_interface=127.6.40.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab
--principal=kudu/127.6.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:35097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.40.126:44775
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:46:51.805224 7812 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:46:51.805477 7812 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:46:51.805519 7812 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:46:51.806705 7812 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:46:51.806780 7812 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 22:46:51.806815 7812 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:46:51.806851 7812 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:46:51.806878 7812 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:46:51.806911 7812 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:46:51.808434 7812 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:35097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.40.126:44775
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab
--principal=kudu/127.6.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.126:44775
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7812
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:46:51.808733 7812 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:46:51.809059 7812 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:46:51.814582 7819 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:51.814672 7821 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:46:51.814805 7812 server_base.cc:1034] running on GCE node
W20260806 22:46:51.814574 7818 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:46:51.815164 7812 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:46:51.815526 7812 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:46:51.816715 7812 hybrid_clock.cc:648] HybridClock initialized: now 1786056411816682 us; error 49 us; skew 500 ppm
Aug 06 22:46:51 dist-test-slave-zh2d krb5kdc[7792](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056411, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:46:51.825217 7812 init.cc:377] Logged in from keytab as kudu/127.6.40.126@KRBTEST.COM (short username kudu)
I20260806 22:46:51.825743 7812 webserver.cc:460] Webserver started at http://127.6.40.126:33231/ using document root <none> and password file <none>
I20260806 22:46:51.826009 7812 fs_manager.cc:359] Metadata directory not provided
I20260806 22:46:51.826082 7812 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:46:51.826236 7812 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:46:51.827399 7812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/minicluster-data/master-0/data/instance:
uuid: "23483819909e4fd5a091b0e031dffb15"
format_stamp: "Formatted at 2026-08-06 22:46:51 on dist-test-slave-zh2d"
server_key: "2fae9accbc31b0020e09ae13e8f42307"
server_key_iv: "e2f61827a196a580ebbe8078277f6915"
server_key_version: "encryptionkey@0"
I20260806 22:46:51.827777 7812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/minicluster-data/master-0/wal/instance:
uuid: "23483819909e4fd5a091b0e031dffb15"
format_stamp: "Formatted at 2026-08-06 22:46:51 on dist-test-slave-zh2d"
server_key: "2fae9accbc31b0020e09ae13e8f42307"
server_key_iv: "e2f61827a196a580ebbe8078277f6915"
server_key_version: "encryptionkey@0"
I20260806 22:46:51.829246 7812 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:46:51.830045 7830 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:51.830216 7812 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:46:51.830327 7812 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/minicluster-data/master-0/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/minicluster-data/master-0/wal
uuid: "23483819909e4fd5a091b0e031dffb15"
format_stamp: "Formatted at 2026-08-06 22:46:51 on dist-test-slave-zh2d"
server_key: "2fae9accbc31b0020e09ae13e8f42307"
server_key_iv: "e2f61827a196a580ebbe8078277f6915"
server_key_version: "encryptionkey@0"
I20260806 22:46:51.830425 7812 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:46:51.848649 7812 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:46:51.851014 7812 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:46:51.851194 7812 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:46:51.856524 7812 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.126:44775
I20260806 22:46:51.856576 7911 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.126:44775 every 8 connection(s)
I20260806 22:46:51.856935 7812 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
I20260806 22:46:51.857612 7912 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:46:51.859972 7912 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23483819909e4fd5a091b0e031dffb15: Bootstrap starting.
I20260806 22:46:51.860746 7912 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 23483819909e4fd5a091b0e031dffb15: Neither blocks nor log segments found. Creating new log.
I20260806 22:46:51.861173 7912 log.cc:824] T 00000000000000000000000000000000 P 23483819909e4fd5a091b0e031dffb15: Log is configured to *not* fsync() on all Append() calls
I20260806 22:46:51.861972 7912 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23483819909e4fd5a091b0e031dffb15: No bootstrap required, opened a new log
I20260806 22:46:51.864292 7912 raft_consensus.cc:348] T 00000000000000000000000000000000 P 23483819909e4fd5a091b0e031dffb15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23483819909e4fd5a091b0e031dffb15" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 44775 } }
I20260806 22:46:51.864439 7912 raft_consensus.cc:374] T 00000000000000000000000000000000 P 23483819909e4fd5a091b0e031dffb15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:46:51.864461 7912 raft_consensus.cc:729] T 00000000000000000000000000000000 P 23483819909e4fd5a091b0e031dffb15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23483819909e4fd5a091b0e031dffb15, State: Initialized, Role: FOLLOWER
I20260806 22:46:51.864635 7912 consensus_queue.cc:260] T 00000000000000000000000000000000 P 23483819909e4fd5a091b0e031dffb15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23483819909e4fd5a091b0e031dffb15" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 44775 } }
I20260806 22:46:51.864708 7912 raft_consensus.cc:388] T 00000000000000000000000000000000 P 23483819909e4fd5a091b0e031dffb15 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:46:51.864732 7912 raft_consensus.cc:482] T 00000000000000000000000000000000 P 23483819909e4fd5a091b0e031dffb15 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:46:51.864799 7912 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 23483819909e4fd5a091b0e031dffb15 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:46:51.865388 7912 raft_consensus.cc:504] T 00000000000000000000000000000000 P 23483819909e4fd5a091b0e031dffb15 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23483819909e4fd5a091b0e031dffb15" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 44775 } }
I20260806 22:46:51.865476 7912 leader_election.cc:302] T 00000000000000000000000000000000 P 23483819909e4fd5a091b0e031dffb15 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 23483819909e4fd5a091b0e031dffb15; no voters:
I20260806 22:46:51.865686 7912 leader_election.cc:288] T 00000000000000000000000000000000 P 23483819909e4fd5a091b0e031dffb15 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:46:51.865779 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 7812
I20260806 22:46:51.865903 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/minicluster-data/master-0/wal/instance
I20260806 22:46:51.865916 7918 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 23483819909e4fd5a091b0e031dffb15 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:46:51.866132 6305 external_mini_cluster.cc:1428] Setting key 0584b0e6961b9a2824238439c2de092d
I20260806 22:46:51.866180 7912 sys_catalog.cc:564] T 00000000000000000000000000000000 P 23483819909e4fd5a091b0e031dffb15 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:46:51.866317 7918 raft_consensus.cc:686] T 00000000000000000000000000000000 P 23483819909e4fd5a091b0e031dffb15 [term 1 LEADER]: Becoming Leader. State: Replica: 23483819909e4fd5a091b0e031dffb15, State: Running, Role: LEADER
I20260806 22:46:51.866472 7918 consensus_queue.cc:237] T 00000000000000000000000000000000 P 23483819909e4fd5a091b0e031dffb15 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23483819909e4fd5a091b0e031dffb15" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 44775 } }
I20260806 22:46:51.866979 7920 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23483819909e4fd5a091b0e031dffb15 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "23483819909e4fd5a091b0e031dffb15" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23483819909e4fd5a091b0e031dffb15" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 44775 } } }
I20260806 22:46:51.867079 7920 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23483819909e4fd5a091b0e031dffb15 [sys.catalog]: This master's current role is: LEADER
I20260806 22:46:51.867249 7921 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23483819909e4fd5a091b0e031dffb15 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 23483819909e4fd5a091b0e031dffb15. Latest consensus state: current_term: 1 leader_uuid: "23483819909e4fd5a091b0e031dffb15" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23483819909e4fd5a091b0e031dffb15" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 44775 } } }
I20260806 22:46:51.867326 7921 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23483819909e4fd5a091b0e031dffb15 [sys.catalog]: This master's current role is: LEADER
I20260806 22:46:51.867401 7931 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:46:51.868010 7931 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:46:51.869956 7931 catalog_manager.cc:1347] Generated new cluster ID: 7d794ce623714cc8bca46d667b168dd7
I20260806 22:46:51.870018 7931 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 22:46:51 dist-test-slave-zh2d krb5kdc[7792](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056411, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
I20260806 22:46:51.876545 7924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:51.866705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44240 (local address 127.6.40.126:44775)
0806 22:46:51.867010 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:51.867021 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:51.867061 (+ 40us) server_negotiation.cc:407] Connection header received
0806 22:46:51.867192 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:51.867200 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:51.867284 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:51.867449 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:51.868317 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:51.869413 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:51.870365 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:51.870784 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:51.870903 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:51.873269 (+ 2366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:51.873273 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:51.873273 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:46:51.873286 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:51.874974 (+ 1688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:51.875359 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:51.875362 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:51.875363 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:51.875469 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:51.875687 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:51.875688 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:51.875689 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:51.875935 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:51.876108 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:51.876186 (+ 78us) server_negotiation.cc:299] Negotiation successful
0806 22:46:51.876330 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":245,"thread_start_us":132,"threads_started":1}
I20260806 22:46:51.880724 7931 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:46:51.881357 7931 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:46:51.884444 7931 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 23483819909e4fd5a091b0e031dffb15: Generated new TSK 0
I20260806 22:46:51.884629 7931 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.40.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.40.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.40.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.65.keytab.
Entry for principal kudu/127.6.40.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.65.keytab.
I20260806 22:46:51.916057 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.65:0
--local_ip_for_outbound_sockets=127.6.40.65
--webserver_interface=127.6.40.65
--webserver_port=0
--tserver_master_addrs=127.6.40.126:44775
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.65.keytab
--principal=kudu/127.6.40.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:35097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:46:52.002887 7950 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:46:52.003134 7950 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:46:52.003178 7950 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:46:52.004416 7950 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:46:52.004487 7950 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:46:52.004525 7950 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 22:46:52.004583 7950 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.65
I20260806 22:46:52.006183 7950 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:35097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.65.keytab
--principal=kudu/127.6.40.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.40.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.40.126:44775
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7950
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.65
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:46:52.006458 7950 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:46:52.006716 7950 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:46:52.012867 7959 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:52.012909 7958 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:46:52.013206 7950 server_base.cc:1034] running on GCE node
W20260806 22:46:52.013307 7962 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:46:52.013553 7950 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:46:52.013808 7950 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:46:52.014974 7950 hybrid_clock.cc:648] HybridClock initialized: now 1786056412014950 us; error 41 us; skew 500 ppm
Aug 06 22:46:52 dist-test-slave-zh2d krb5kdc[7792](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056412, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:46:52.017730 7950 init.cc:377] Logged in from keytab as kudu/127.6.40.65@KRBTEST.COM (short username kudu)
I20260806 22:46:52.018250 7950 webserver.cc:460] Webserver started at http://127.6.40.65:39221/ using document root <none> and password file <none>
I20260806 22:46:52.018501 7950 fs_manager.cc:359] Metadata directory not provided
I20260806 22:46:52.018571 7950 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:46:52.018723 7950 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:46:52.019801 7950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/minicluster-data/ts-0/data/instance:
uuid: "9ab7e43baddc4a6aaa3ef480a35e0896"
format_stamp: "Formatted at 2026-08-06 22:46:52 on dist-test-slave-zh2d"
server_key: "5770ec101e7640897fc70cd8f5b33e4e"
server_key_iv: "a77bb1ff9dd20354826d2812cc7d2ac1"
server_key_version: "encryptionkey@0"
I20260806 22:46:52.020171 7950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/minicluster-data/ts-0/wal/instance:
uuid: "9ab7e43baddc4a6aaa3ef480a35e0896"
format_stamp: "Formatted at 2026-08-06 22:46:52 on dist-test-slave-zh2d"
server_key: "5770ec101e7640897fc70cd8f5b33e4e"
server_key_iv: "a77bb1ff9dd20354826d2812cc7d2ac1"
server_key_version: "encryptionkey@0"
I20260806 22:46:52.021606 7950 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:46:52.022424 7975 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:52.022615 7950 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:46:52.022707 7950 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/minicluster-data/ts-0/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/minicluster-data/ts-0/wal
uuid: "9ab7e43baddc4a6aaa3ef480a35e0896"
format_stamp: "Formatted at 2026-08-06 22:46:52 on dist-test-slave-zh2d"
server_key: "5770ec101e7640897fc70cd8f5b33e4e"
server_key_iv: "a77bb1ff9dd20354826d2812cc7d2ac1"
server_key_version: "encryptionkey@0"
I20260806 22:46:52.022787 7950 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:46:52.031160 7950 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:46:52.033028 7950 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:46:52.033217 7950 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:46:52.033773 7950 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:46:52.033836 7950 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:52.033874 7950 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:46:52.033902 7950 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 22:46:52 dist-test-slave-zh2d krb5kdc[7792](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056412, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.65@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
I20260806 22:46:52.041667 7950 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.65:43723
I20260806 22:46:52.041745 8138 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.65:43723 every 8 connection(s)
I20260806 22:46:52.042095 7950 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
I20260806 22:46:52.044139 7924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:52.034297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.65:38451 (local address 127.6.40.126:44775)
0806 22:46:52.034339 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:52.034342 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:52.034582 (+ 240us) server_negotiation.cc:407] Connection header received
0806 22:46:52.035058 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:52.035059 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:52.035077 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:52.035159 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:52.035814 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:52.036394 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:52.037621 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:52.037840 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:52.037933 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:52.041340 (+ 3407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:52.041342 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:52.041342 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:46:52.041354 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:52.043044 (+ 1690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:52.043405 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:52.043406 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:52.043406 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:52.043486 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:52.043669 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:52.043669 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:52.043670 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:52.043762 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:52.043807 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:52.044001 (+ 194us) server_negotiation.cc:299] Negotiation successful
0806 22:46:52.044074 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 22:46:52.044277 7987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:52.034264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:44775 (local address 127.6.40.65:38451)
0806 22:46:52.034453 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:52.034490 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:52.034954 (+ 464us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:52.035206 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:52.035210 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:52.035450 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:46:52.035718 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:52.035726 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:52.036495 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:52.036496 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:52.037521 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:52.037524 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:52.037952 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:52.037954 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:52.038083 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:52.038604 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:46:52.038616 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:46:52.041241 (+ 2625us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:46:52.043118 (+ 1877us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:52.043124 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:52.043126 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:52.043335 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:52.043545 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:52.043546 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:52.043546 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:52.043632 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:52.043796 (+ 164us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:46:52.043798 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:46:52.043906 (+ 108us) client_negotiation.cc:769] Sending connection context
0806 22:46:52.043978 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 22:46:52.044094 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":94,"threads_started":1}
I20260806 22:46:52.044858 8141 heartbeater.cc:347] Connected to a master server at 127.6.40.126:44775
I20260806 22:46:52.044970 8141 heartbeater.cc:464] Registering TS with master...
I20260806 22:46:52.045145 8141 heartbeater.cc:511] Master 127.6.40.126:44775 requested a full tablet report, sending...
I20260806 22:46:52.045701 7849 ts_manager.cc:194] Registered new tserver with Master: 9ab7e43baddc4a6aaa3ef480a35e0896 (127.6.40.65:43723)
I20260806 22:46:52.046317 7849 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.40.65@KRBTEST.COM'} at 127.6.40.65:38451
I20260806 22:46:52.051764 7924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:52.046687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.65:36053 (local address 127.6.40.126:44775)
0806 22:46:52.046725 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:52.046727 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:52.046840 (+ 113us) server_negotiation.cc:407] Connection header received
0806 22:46:52.046867 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:52.046868 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:52.046878 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:52.046948 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:52.047235 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:52.047720 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:52.048458 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:52.048633 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:52.048682 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:52.049599 (+ 917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:52.049600 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:52.049600 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:46:52.049606 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:52.050709 (+ 1103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:52.051099 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:52.051100 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:52.051100 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:52.051147 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:52.051340 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:52.051340 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:52.051340 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:52.051424 (+ 84us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:52.051496 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:52.051649 (+ 153us) server_negotiation.cc:299] Negotiation successful
0806 22:46:52.051707 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260806 22:46:52.051813 8147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:52.046624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:44775 (local address 127.6.40.65:36053)
0806 22:46:52.046764 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:52.046774 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:52.046833 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:52.046949 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:52.046950 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:52.047106 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:46:52.047165 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:52.047167 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:52.047815 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:52.047816 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:52.048361 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:52.048362 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:52.048696 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:52.048697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:52.048753 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:52.049225 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:46:52.049231 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:46:52.049524 (+ 293us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:46:52.050817 (+ 1293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:52.050818 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:52.050819 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:52.051017 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:52.051208 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:52.051209 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:52.051209 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:52.051275 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:52.051507 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:46:52.051508 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:46:52.051563 (+ 55us) client_negotiation.cc:769] Sending connection context
0806 22:46:52.051633 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 22:46:52.051723 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":80,"threads_started":1}
I20260806 22:46:52.052208 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 7950
I20260806 22:46:52.052309 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/minicluster-data/ts-0/wal/instance
I20260806 22:46:52.052475 6305 external_mini_cluster.cc:1428] Setting key 7d5ac63a345c6aa355ed26f2df991464
WARNING: no policy specified for kudu/127.6.40.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.40.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.40.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.66.keytab.
Entry for principal kudu/127.6.40.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.66.keytab.
I20260806 22:46:52.079311 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.66:0
--local_ip_for_outbound_sockets=127.6.40.66
--webserver_interface=127.6.40.66
--webserver_port=0
--tserver_master_addrs=127.6.40.126:44775
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.66.keytab
--principal=kudu/127.6.40.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:35097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:46:52.165211 8150 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:46:52.165437 8150 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:46:52.165479 8150 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:46:52.166698 8150 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:46:52.166754 8150 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:46:52.166791 8150 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 22:46:52.166846 8150 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.66
I20260806 22:46:52.168358 8150 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:35097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.66.keytab
--principal=kudu/127.6.40.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.40.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.40.126:44775
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8150
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.66
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:46:52.168617 8150 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:46:52.168889 8150 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:46:52.174026 8158 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:52.173992 8156 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:52.174152 8155 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:46:52.174361 8150 server_base.cc:1034] running on GCE node
I20260806 22:46:52.174593 8150 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:46:52.174825 8150 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:46:52.176033 8150 hybrid_clock.cc:648] HybridClock initialized: now 1786056412176017 us; error 26 us; skew 500 ppm
Aug 06 22:46:52 dist-test-slave-zh2d krb5kdc[7792](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056412, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:46:52.178702 8150 init.cc:377] Logged in from keytab as kudu/127.6.40.66@KRBTEST.COM (short username kudu)
I20260806 22:46:52.179217 8150 webserver.cc:460] Webserver started at http://127.6.40.66:33233/ using document root <none> and password file <none>
I20260806 22:46:52.179467 8150 fs_manager.cc:359] Metadata directory not provided
I20260806 22:46:52.179536 8150 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:46:52.179683 8150 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:46:52.180723 8150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/minicluster-data/ts-1/data/instance:
uuid: "690eb04bf30e4fadac3f1b2ac9125758"
format_stamp: "Formatted at 2026-08-06 22:46:52 on dist-test-slave-zh2d"
server_key: "af59a659c96a7fecdf99d58bc1578ff6"
server_key_iv: "91d3c78be27e02db2eba6d4bfa4d05e2"
server_key_version: "encryptionkey@0"
I20260806 22:46:52.181161 8150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/minicluster-data/ts-1/wal/instance:
uuid: "690eb04bf30e4fadac3f1b2ac9125758"
format_stamp: "Formatted at 2026-08-06 22:46:52 on dist-test-slave-zh2d"
server_key: "af59a659c96a7fecdf99d58bc1578ff6"
server_key_iv: "91d3c78be27e02db2eba6d4bfa4d05e2"
server_key_version: "encryptionkey@0"
I20260806 22:46:52.182577 8150 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:46:52.183343 8167 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:52.183521 8150 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:46:52.183627 8150 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/minicluster-data/ts-1/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/minicluster-data/ts-1/wal
uuid: "690eb04bf30e4fadac3f1b2ac9125758"
format_stamp: "Formatted at 2026-08-06 22:46:52 on dist-test-slave-zh2d"
server_key: "af59a659c96a7fecdf99d58bc1578ff6"
server_key_iv: "91d3c78be27e02db2eba6d4bfa4d05e2"
server_key_version: "encryptionkey@0"
I20260806 22:46:52.183701 8150 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:46:52.195554 8150 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:46:52.197937 8150 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:46:52.198097 8150 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:46:52.198666 8150 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:46:52.198722 8150 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:52.198763 8150 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:46:52.198786 8150 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 22:46:52 dist-test-slave-zh2d krb5kdc[7792](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056412, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.66@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
I20260806 22:46:52.206647 8150 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.66:40507
I20260806 22:46:52.206733 8323 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.66:40507 every 8 connection(s)
I20260806 22:46:52.207079 8150 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
I20260806 22:46:52.209354 7924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:52.199060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.66:47035 (local address 127.6.40.126:44775)
0806 22:46:52.199137 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:52.199140 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:52.199390 (+ 250us) server_negotiation.cc:407] Connection header received
0806 22:46:52.199944 (+ 554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:52.199945 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:52.199960 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:52.200013 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:52.200802 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:52.201730 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:52.202838 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:52.203082 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:52.203166 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:52.206564 (+ 3398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:52.206565 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:52.206566 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:46:52.206575 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:52.208094 (+ 1519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:52.208496 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:52.208497 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:52.208498 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:52.208554 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:52.208773 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:52.208774 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:52.208774 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:52.208938 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:52.208994 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:52.209221 (+ 227us) server_negotiation.cc:299] Negotiation successful
0806 22:46:52.209290 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 22:46:52.209568 8180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:52.199044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:44775 (local address 127.6.40.66:47035)
0806 22:46:52.199242 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:52.199282 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:52.199831 (+ 549us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:52.200081 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:52.200090 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:52.200369 (+ 279us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:46:52.200687 (+ 318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:52.200694 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:52.201840 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:52.201841 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:52.202747 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:52.202750 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:52.203174 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:52.203176 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:52.203310 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:52.203854 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:46:52.203864 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:46:52.206464 (+ 2600us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:46:52.208167 (+ 1703us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:52.208172 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:52.208174 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:52.208419 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:52.208617 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:52.208618 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:52.208618 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:52.208715 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:52.208995 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:46:52.208999 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:46:52.209109 (+ 110us) client_negotiation.cc:769] Sending connection context
0806 22:46:52.209186 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 22:46:52.209313 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":127,"threads_started":1}
I20260806 22:46:52.210059 8324 heartbeater.cc:347] Connected to a master server at 127.6.40.126:44775
I20260806 22:46:52.210141 8324 heartbeater.cc:464] Registering TS with master...
I20260806 22:46:52.210325 8324 heartbeater.cc:511] Master 127.6.40.126:44775 requested a full tablet report, sending...
I20260806 22:46:52.210711 7850 ts_manager.cc:194] Registered new tserver with Master: 690eb04bf30e4fadac3f1b2ac9125758 (127.6.40.66:40507)
I20260806 22:46:52.211241 7850 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.40.66@KRBTEST.COM'} at 127.6.40.66:47035
I20260806 22:46:52.214476 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 8150
I20260806 22:46:52.214576 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/minicluster-data/ts-1/wal/instance
I20260806 22:46:52.214802 6305 external_mini_cluster.cc:1428] Setting key 85738c73e34055c6f5b3ffa1eb7da5dc
I20260806 22:46:52.217191 8332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:52.211135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:44775 (local address 127.6.40.66:54431)
0806 22:46:52.211280 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:52.211290 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:52.211358 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:52.211490 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:52.211491 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:52.211703 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:46:52.211786 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:52.211788 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:52.212512 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:52.212513 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:52.213146 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:52.213148 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:52.213468 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:52.213469 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:52.213529 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:52.213959 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:46:52.213964 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:46:52.214245 (+ 281us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:46:52.216031 (+ 1786us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:52.216032 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:52.216032 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:52.216193 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:52.216397 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:52.216398 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:52.216398 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:52.216454 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:52.216795 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:46:52.216796 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:46:52.216868 (+ 72us) client_negotiation.cc:769] Sending connection context
0806 22:46:52.216940 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 22:46:52.217045 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"mutex_wait_us":1,"thread_start_us":73,"threads_started":1}
I20260806 22:46:52.217202 7924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:52.211262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.66:54431 (local address 127.6.40.126:44775)
0806 22:46:52.211365 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:52.211367 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:52.211378 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:46:52.211410 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:52.211411 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:52.211420 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:52.211479 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:52.211867 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:52.212404 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:52.213214 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:52.213395 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:52.213466 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:52.214332 (+ 866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:52.214333 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:52.214334 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:46:52.214341 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:52.215953 (+ 1612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:52.216265 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:52.216266 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:52.216266 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:52.216330 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:52.216518 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:52.216519 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:52.216519 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:52.216724 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:52.216797 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:52.216982 (+ 185us) server_negotiation.cc:299] Negotiation successful
0806 22:46:52.217067 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
WARNING: no policy specified for kudu/127.6.40.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.40.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.40.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.67.keytab.
Entry for principal kudu/127.6.40.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.67.keytab.
I20260806 22:46:52.243609 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.67:0
--local_ip_for_outbound_sockets=127.6.40.67
--webserver_interface=127.6.40.67
--webserver_port=0
--tserver_master_addrs=127.6.40.126:44775
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.67.keytab
--principal=kudu/127.6.40.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:35097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:46:52.328884 8335 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:46:52.329082 8335 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:46:52.329106 8335 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:46:52.330291 8335 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:46:52.330331 8335 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:46:52.330351 8335 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 22:46:52.330395 8335 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.67
I20260806 22:46:52.331815 8335 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:35097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.67.keytab
--principal=kudu/127.6.40.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.40.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.40.126:44775
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8335
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.67
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:46:52.332067 8335 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:46:52.332293 8335 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:46:52.337491 8346 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:52.337483 8343 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:52.337555 8344 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:46:52.337698 8335 server_base.cc:1034] running on GCE node
I20260806 22:46:52.337953 8335 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:46:52.338199 8335 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:46:52.339360 8335 hybrid_clock.cc:648] HybridClock initialized: now 1786056412339353 us; error 39 us; skew 500 ppm
Aug 06 22:46:52 dist-test-slave-zh2d krb5kdc[7792](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056412, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:46:52.342095 8335 init.cc:377] Logged in from keytab as kudu/127.6.40.67@KRBTEST.COM (short username kudu)
I20260806 22:46:52.342546 8335 webserver.cc:460] Webserver started at http://127.6.40.67:37669/ using document root <none> and password file <none>
I20260806 22:46:52.342800 8335 fs_manager.cc:359] Metadata directory not provided
I20260806 22:46:52.342869 8335 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:46:52.343043 8335 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:46:52.344187 8335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/minicluster-data/ts-2/data/instance:
uuid: "fab98111c13547c8895666985780a499"
format_stamp: "Formatted at 2026-08-06 22:46:52 on dist-test-slave-zh2d"
server_key: "84cbb57200e9b39e3f60220a6c322f94"
server_key_iv: "b2506f92f3014d2982dd6e9585708fe7"
server_key_version: "encryptionkey@0"
I20260806 22:46:52.344560 8335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/minicluster-data/ts-2/wal/instance:
uuid: "fab98111c13547c8895666985780a499"
format_stamp: "Formatted at 2026-08-06 22:46:52 on dist-test-slave-zh2d"
server_key: "84cbb57200e9b39e3f60220a6c322f94"
server_key_iv: "b2506f92f3014d2982dd6e9585708fe7"
server_key_version: "encryptionkey@0"
I20260806 22:46:52.346074 8335 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:46:52.346889 8357 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:52.347092 8335 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:46:52.347204 8335 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/minicluster-data/ts-2/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/minicluster-data/ts-2/wal
uuid: "fab98111c13547c8895666985780a499"
format_stamp: "Formatted at 2026-08-06 22:46:52 on dist-test-slave-zh2d"
server_key: "84cbb57200e9b39e3f60220a6c322f94"
server_key_iv: "b2506f92f3014d2982dd6e9585708fe7"
server_key_version: "encryptionkey@0"
I20260806 22:46:52.347282 8335 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:46:52.355338 8335 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:46:52.357797 8335 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:46:52.357967 8335 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:46:52.358500 8335 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:46:52.358568 8335 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:52.358618 8335 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:46:52.358661 8335 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 22:46:52 dist-test-slave-zh2d krb5kdc[7792](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056412, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.67@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
I20260806 22:46:52.366860 8335 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.67:45667
I20260806 22:46:52.366969 8524 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.67:45667 every 8 connection(s)
I20260806 22:46:52.367337 8335 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
I20260806 22:46:52.369143 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 8335
I20260806 22:46:52.369254 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/minicluster-data/ts-2/wal/instance
I20260806 22:46:52.369431 6305 external_mini_cluster.cc:1428] Setting key aee19f582ac399b4154a0820461805be
I20260806 22:46:52.370492 7924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:52.359107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.67:37867 (local address 127.6.40.126:44775)
0806 22:46:52.359188 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:52.359191 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:52.359412 (+ 221us) server_negotiation.cc:407] Connection header received
0806 22:46:52.359975 (+ 563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:52.359976 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:52.359991 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:52.360049 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:52.361129 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:52.361974 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:52.363752 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:52.364100 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:52.364178 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:52.367121 (+ 2943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:52.367123 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:52.367123 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:46:52.367130 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:52.368802 (+ 1672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:52.369296 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:52.369297 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:52.369298 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:52.369369 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:52.369698 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:52.369700 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:52.369700 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:52.369857 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:52.369952 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:52.370300 (+ 348us) server_negotiation.cc:299] Negotiation successful
0806 22:46:52.370403 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 22:46:52.370692 8367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:52.359070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:44775 (local address 127.6.40.67:37867)
0806 22:46:52.359269 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:52.359310 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:52.359843 (+ 533us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:52.360193 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:52.360198 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:52.360643 (+ 445us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:46:52.360990 (+ 347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:52.360998 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:52.362102 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:52.362104 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:52.363654 (+ 1550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:52.363658 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:52.364222 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:52.364223 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:52.364403 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:52.364969 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:46:52.364982 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:46:52.366945 (+ 1963us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:46:52.368950 (+ 2005us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:52.368957 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:52.368959 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:52.369191 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:52.369438 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:52.369439 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:52.369439 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:52.369594 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:52.369940 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:46:52.369948 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:46:52.370140 (+ 192us) client_negotiation.cc:769] Sending connection context
0806 22:46:52.370256 (+ 116us) client_negotiation.cc:240] Negotiation successful
0806 22:46:52.370429 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":101,"threads_started":1}
I20260806 22:46:52.371208 8525 heartbeater.cc:347] Connected to a master server at 127.6.40.126:44775
I20260806 22:46:52.371295 8525 heartbeater.cc:464] Registering TS with master...
I20260806 22:46:52.371479 8525 heartbeater.cc:511] Master 127.6.40.126:44775 requested a full tablet report, sending...
I20260806 22:46:52.372246 7850 ts_manager.cc:194] Registered new tserver with Master: fab98111c13547c8895666985780a499 (127.6.40.67:45667)
I20260806 22:46:52.372803 7850 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.40.67@KRBTEST.COM'} at 127.6.40.67:37867
I20260806 22:46:52.377820 7924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:52.372342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.67:49127 (local address 127.6.40.126:44775)
0806 22:46:52.372415 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:52.372417 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:52.372506 (+ 89us) server_negotiation.cc:407] Connection header received
0806 22:46:52.372577 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:52.372577 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:52.372585 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:52.372626 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:52.373007 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:52.373475 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:52.374182 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:52.374388 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:52.374473 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:52.375391 (+ 918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:52.375392 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:52.375393 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:46:52.375400 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:52.376529 (+ 1129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:52.376938 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:52.376939 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:52.376939 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:52.377012 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:52.377272 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:52.377273 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:52.377274 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:52.377399 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:52.377458 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:52.377652 (+ 194us) server_negotiation.cc:299] Negotiation successful
0806 22:46:52.377737 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 22:46:52.377830 8532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:52.372285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:44775 (local address 127.6.40.67:49127)
0806 22:46:52.372452 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:52.372462 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:52.372522 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:52.372640 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:52.372640 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:52.372865 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:46:52.372941 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:52.372943 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:52.373581 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:52.373582 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:52.374097 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:52.374098 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:52.374477 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:52.374478 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:52.374555 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:52.374991 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:46:52.374996 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:46:52.375282 (+ 286us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:46:52.376629 (+ 1347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:52.376630 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:52.376631 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:52.376877 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:52.377097 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:52.377098 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:52.377099 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:52.377158 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:52.377481 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:46:52.377482 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:46:52.377525 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 22:46:52.377616 (+ 91us) client_negotiation.cc:240] Negotiation successful
0806 22:46:52.377723 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":68,"threads_started":1}
I20260806 22:46:52.381469 6305 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 22:46:52 dist-test-slave-zh2d krb5kdc[7792](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056412, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 22:46:52 dist-test-slave-zh2d krb5kdc[7792](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056412, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
I20260806 22:46:52.405253 7924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:52.398396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44256 (local address 127.6.40.126:44775)
0806 22:46:52.398468 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:52.398470 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:52.398479 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:46:52.398503 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:52.398503 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:52.398512 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:52.398561 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:52.399113 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:52.399598 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:52.400296 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:52.400478 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:52.400527 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:52.402765 (+ 2238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:52.402766 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:52.402766 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:46:52.402773 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:52.404146 (+ 1373us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:52.404483 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:52.404484 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:52.404484 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:52.404541 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:52.404711 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:52.404712 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:52.404712 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:52.404812 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:52.405003 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:52.405123 (+ 120us) server_negotiation.cc:299] Negotiation successful
0806 22:46:52.405190 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 22:46:52.407066 7850 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44256:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:46:52.408689 7850 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:46:52.415607 8548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:52.411062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.67:45667 (local address 127.0.0.1:51258)
0806 22:46:52.411817 (+ 755us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:52.411828 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:52.411871 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:52.412136 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:52.412137 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:52.412140 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:46:52.412192 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:52.412194 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:52.413422 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:52.413423 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:52.414310 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:52.414313 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:52.415254 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:52.415255 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:52.415350 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:52.415372 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 22:46:52.415450 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 22:46:52.415507 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":733,"thread_start_us":92,"threads_started":1}
I20260806 22:46:52.415637 8553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:52.411637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51258 (local address 127.6.40.67:45667)
0806 22:46:52.411992 (+ 355us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:52.411995 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:52.412003 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:46:52.412035 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:52.412037 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:52.412049 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:52.412108 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:46:52.412286 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:52.413326 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:52.414407 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:52.415160 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:52.415241 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:52.415307 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:52.415479 (+ 172us) server_negotiation.cc:299] Negotiation successful
0806 22:46:52.415526 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":86,"threads_started":1}
I20260806 22:46:52.415704 8546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:52.411054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.65:43723 (local address 127.0.0.1:40912)
0806 22:46:52.411342 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:52.411385 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:52.411461 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:52.411697 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:52.411700 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:52.411703 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:46:52.411771 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:52.411774 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:52.413145 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:52.413146 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:52.414417 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:52.414420 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:52.415370 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:52.415372 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:52.415456 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:52.415460 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:46:52.415515 (+ 55us) client_negotiation.cc:240] Negotiation successful
0806 22:46:52.415556 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"mutex_wait_us":3,"thread_start_us":78,"threads_started":1}
I20260806 22:46:52.416071 8550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:52.411151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40912 (local address 127.6.40.65:43723)
0806 22:46:52.411367 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:52.411373 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:52.411439 (+ 66us) server_negotiation.cc:407] Connection header received
0806 22:46:52.411545 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:52.411548 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:52.411563 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:52.411686 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:46:52.411896 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:52.413055 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:52.414517 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:52.415225 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:52.415285 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:52.415335 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:52.415653 (+ 318us) server_negotiation.cc:299] Negotiation successful
0806 22:46:52.415704 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":86,"threads_started":1}
I20260806 22:46:52.416245 8547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:52.411064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.66:40507 (local address 127.0.0.1:34104)
0806 22:46:52.412187 (+ 1123us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:52.412195 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:52.412265 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:52.412447 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:52.412448 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:52.412449 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:46:52.412490 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:52.412492 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:52.413725 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:52.413726 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:52.414704 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:52.414706 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:52.415909 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:52.415910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:52.415976 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:52.415980 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:46:52.416044 (+ 64us) client_negotiation.cc:240] Negotiation successful
0806 22:46:52.416094 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1107,"thread_start_us":107,"threads_started":1}
I20260806 22:46:52.416297 8551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:52.411334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34104 (local address 127.6.40.66:40507)
0806 22:46:52.411586 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:52.411591 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:52.412268 (+ 677us) server_negotiation.cc:407] Connection header received
0806 22:46:52.412341 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:52.412343 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:52.412359 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:52.412450 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:46:52.412550 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:52.413633 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:52.414792 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:52.415382 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:52.415471 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:52.415530 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:52.416106 (+ 576us) server_negotiation.cc:299] Negotiation successful
0806 22:46:52.416185 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":212,"thread_start_us":171,"threads_started":1}
I20260806 22:46:52.416831 8049 tablet_service.cc:1467] Processing CreateTablet for tablet 5c5bbfa118924e11ba1b556c518310e2 (DEFAULT_TABLE table=test-table [id=2554a0a79f264a40b0c61fd78126a7b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:46:52.416992 8431 tablet_service.cc:1467] Processing CreateTablet for tablet 5c5bbfa118924e11ba1b556c518310e2 (DEFAULT_TABLE table=test-table [id=2554a0a79f264a40b0c61fd78126a7b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:46:52.417281 8049 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c5bbfa118924e11ba1b556c518310e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:46:52.417294 8431 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c5bbfa118924e11ba1b556c518310e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:46:52.417524 8237 tablet_service.cc:1467] Processing CreateTablet for tablet 5c5bbfa118924e11ba1b556c518310e2 (DEFAULT_TABLE table=test-table [id=2554a0a79f264a40b0c61fd78126a7b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:46:52.417856 8237 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c5bbfa118924e11ba1b556c518310e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:46:52.420756 8555 tablet_bootstrap.cc:492] T 5c5bbfa118924e11ba1b556c518310e2 P 690eb04bf30e4fadac3f1b2ac9125758: Bootstrap starting.
I20260806 22:46:52.421283 8557 tablet_bootstrap.cc:492] T 5c5bbfa118924e11ba1b556c518310e2 P 9ab7e43baddc4a6aaa3ef480a35e0896: Bootstrap starting.
I20260806 22:46:52.421563 8555 tablet_bootstrap.cc:654] T 5c5bbfa118924e11ba1b556c518310e2 P 690eb04bf30e4fadac3f1b2ac9125758: Neither blocks nor log segments found. Creating new log.
I20260806 22:46:52.421927 8555 log.cc:824] T 5c5bbfa118924e11ba1b556c518310e2 P 690eb04bf30e4fadac3f1b2ac9125758: Log is configured to *not* fsync() on all Append() calls
I20260806 22:46:52.421964 8557 tablet_bootstrap.cc:654] T 5c5bbfa118924e11ba1b556c518310e2 P 9ab7e43baddc4a6aaa3ef480a35e0896: Neither blocks nor log segments found. Creating new log.
I20260806 22:46:52.422314 8557 log.cc:824] T 5c5bbfa118924e11ba1b556c518310e2 P 9ab7e43baddc4a6aaa3ef480a35e0896: Log is configured to *not* fsync() on all Append() calls
I20260806 22:46:52.422334 8556 tablet_bootstrap.cc:492] T 5c5bbfa118924e11ba1b556c518310e2 P fab98111c13547c8895666985780a499: Bootstrap starting.
I20260806 22:46:52.422758 8555 tablet_bootstrap.cc:492] T 5c5bbfa118924e11ba1b556c518310e2 P 690eb04bf30e4fadac3f1b2ac9125758: No bootstrap required, opened a new log
I20260806 22:46:52.422837 8555 ts_tablet_manager.cc:1397] T 5c5bbfa118924e11ba1b556c518310e2 P 690eb04bf30e4fadac3f1b2ac9125758: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:46:52.423161 8557 tablet_bootstrap.cc:492] T 5c5bbfa118924e11ba1b556c518310e2 P 9ab7e43baddc4a6aaa3ef480a35e0896: No bootstrap required, opened a new log
I20260806 22:46:52.423240 8557 ts_tablet_manager.cc:1397] T 5c5bbfa118924e11ba1b556c518310e2 P 9ab7e43baddc4a6aaa3ef480a35e0896: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:46:52.423317 8556 tablet_bootstrap.cc:654] T 5c5bbfa118924e11ba1b556c518310e2 P fab98111c13547c8895666985780a499: Neither blocks nor log segments found. Creating new log.
I20260806 22:46:52.423756 8556 log.cc:824] T 5c5bbfa118924e11ba1b556c518310e2 P fab98111c13547c8895666985780a499: Log is configured to *not* fsync() on all Append() calls
I20260806 22:46:52.424710 8556 tablet_bootstrap.cc:492] T 5c5bbfa118924e11ba1b556c518310e2 P fab98111c13547c8895666985780a499: No bootstrap required, opened a new log
I20260806 22:46:52.424788 8556 ts_tablet_manager.cc:1397] T 5c5bbfa118924e11ba1b556c518310e2 P fab98111c13547c8895666985780a499: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 22:46:52.424830 8555 raft_consensus.cc:348] T 5c5bbfa118924e11ba1b556c518310e2 P 690eb04bf30e4fadac3f1b2ac9125758 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ab7e43baddc4a6aaa3ef480a35e0896" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 43723 } } peers { permanent_uuid: "fab98111c13547c8895666985780a499" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45667 } } peers { permanent_uuid: "690eb04bf30e4fadac3f1b2ac9125758" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 40507 } }
I20260806 22:46:52.425068 8555 raft_consensus.cc:374] T 5c5bbfa118924e11ba1b556c518310e2 P 690eb04bf30e4fadac3f1b2ac9125758 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:46:52.425114 8555 raft_consensus.cc:729] T 5c5bbfa118924e11ba1b556c518310e2 P 690eb04bf30e4fadac3f1b2ac9125758 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 690eb04bf30e4fadac3f1b2ac9125758, State: Initialized, Role: FOLLOWER
I20260806 22:46:52.425266 8555 consensus_queue.cc:260] T 5c5bbfa118924e11ba1b556c518310e2 P 690eb04bf30e4fadac3f1b2ac9125758 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ab7e43baddc4a6aaa3ef480a35e0896" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 43723 } } peers { permanent_uuid: "fab98111c13547c8895666985780a499" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45667 } } peers { permanent_uuid: "690eb04bf30e4fadac3f1b2ac9125758" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 40507 } }
I20260806 22:46:52.425578 8324 heartbeater.cc:503] Master 127.6.40.126:44775 was elected leader, sending a full tablet report...
I20260806 22:46:52.425664 8557 raft_consensus.cc:348] T 5c5bbfa118924e11ba1b556c518310e2 P 9ab7e43baddc4a6aaa3ef480a35e0896 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ab7e43baddc4a6aaa3ef480a35e0896" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 43723 } } peers { permanent_uuid: "fab98111c13547c8895666985780a499" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45667 } } peers { permanent_uuid: "690eb04bf30e4fadac3f1b2ac9125758" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 40507 } }
I20260806 22:46:52.425837 8555 ts_tablet_manager.cc:1428] T 5c5bbfa118924e11ba1b556c518310e2 P 690eb04bf30e4fadac3f1b2ac9125758: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260806 22:46:52.425839 8557 raft_consensus.cc:374] T 5c5bbfa118924e11ba1b556c518310e2 P 9ab7e43baddc4a6aaa3ef480a35e0896 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:46:52.426362 8557 raft_consensus.cc:729] T 5c5bbfa118924e11ba1b556c518310e2 P 9ab7e43baddc4a6aaa3ef480a35e0896 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ab7e43baddc4a6aaa3ef480a35e0896, State: Initialized, Role: FOLLOWER
I20260806 22:46:52.426522 8557 consensus_queue.cc:260] T 5c5bbfa118924e11ba1b556c518310e2 P 9ab7e43baddc4a6aaa3ef480a35e0896 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ab7e43baddc4a6aaa3ef480a35e0896" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 43723 } } peers { permanent_uuid: "fab98111c13547c8895666985780a499" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45667 } } peers { permanent_uuid: "690eb04bf30e4fadac3f1b2ac9125758" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 40507 } }
I20260806 22:46:52.426841 8141 heartbeater.cc:503] Master 127.6.40.126:44775 was elected leader, sending a full tablet report...
I20260806 22:46:52.426831 8557 ts_tablet_manager.cc:1428] T 5c5bbfa118924e11ba1b556c518310e2 P 9ab7e43baddc4a6aaa3ef480a35e0896: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260806 22:46:52.427057 8556 raft_consensus.cc:348] T 5c5bbfa118924e11ba1b556c518310e2 P fab98111c13547c8895666985780a499 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ab7e43baddc4a6aaa3ef480a35e0896" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 43723 } } peers { permanent_uuid: "fab98111c13547c8895666985780a499" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45667 } } peers { permanent_uuid: "690eb04bf30e4fadac3f1b2ac9125758" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 40507 } }
I20260806 22:46:52.427218 8556 raft_consensus.cc:374] T 5c5bbfa118924e11ba1b556c518310e2 P fab98111c13547c8895666985780a499 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:46:52.427265 8556 raft_consensus.cc:729] T 5c5bbfa118924e11ba1b556c518310e2 P fab98111c13547c8895666985780a499 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fab98111c13547c8895666985780a499, State: Initialized, Role: FOLLOWER
I20260806 22:46:52.427393 8556 consensus_queue.cc:260] T 5c5bbfa118924e11ba1b556c518310e2 P fab98111c13547c8895666985780a499 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ab7e43baddc4a6aaa3ef480a35e0896" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 43723 } } peers { permanent_uuid: "fab98111c13547c8895666985780a499" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45667 } } peers { permanent_uuid: "690eb04bf30e4fadac3f1b2ac9125758" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 40507 } }
I20260806 22:46:52.427680 8556 ts_tablet_manager.cc:1428] T 5c5bbfa118924e11ba1b556c518310e2 P fab98111c13547c8895666985780a499: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:46:52.427731 8525 heartbeater.cc:503] Master 127.6.40.126:44775 was elected leader, sending a full tablet report...
W20260806 22:46:52.457953 8325 tablet.cc:2369] T 5c5bbfa118924e11ba1b556c518310e2 P 690eb04bf30e4fadac3f1b2ac9125758: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:46:52.542970 8142 tablet.cc:2369] T 5c5bbfa118924e11ba1b556c518310e2 P 9ab7e43baddc4a6aaa3ef480a35e0896: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:46:52.618181 8526 tablet.cc:2369] T 5c5bbfa118924e11ba1b556c518310e2 P fab98111c13547c8895666985780a499: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:46:52.668411 8564 raft_consensus.cc:482] T 5c5bbfa118924e11ba1b556c518310e2 P 9ab7e43baddc4a6aaa3ef480a35e0896 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:46:52.668555 8564 raft_consensus.cc:504] T 5c5bbfa118924e11ba1b556c518310e2 P 9ab7e43baddc4a6aaa3ef480a35e0896 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ab7e43baddc4a6aaa3ef480a35e0896" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 43723 } } peers { permanent_uuid: "fab98111c13547c8895666985780a499" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45667 } } peers { permanent_uuid: "690eb04bf30e4fadac3f1b2ac9125758" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 40507 } }
I20260806 22:46:52.669039 8564 leader_election.cc:288] T 5c5bbfa118924e11ba1b556c518310e2 P 9ab7e43baddc4a6aaa3ef480a35e0896 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fab98111c13547c8895666985780a499 (127.6.40.67:45667), 690eb04bf30e4fadac3f1b2ac9125758 (127.6.40.66:40507)
I20260806 22:46:52.672216 8553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:52.669431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.65:49965 (local address 127.6.40.67:45667)
0806 22:46:52.669496 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:52.669499 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:52.669511 (+ 12us) server_negotiation.cc:407] Connection header received
0806 22:46:52.669598 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:52.669599 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:52.669613 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:52.669672 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:46:52.669874 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:52.670368 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:52.671471 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:52.671823 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:52.671879 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:52.671888 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:52.672088 (+ 200us) server_negotiation.cc:299] Negotiation successful
0806 22:46:52.672141 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 22:46:52.672240 8567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:52.669232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.67:45667 (local address 127.6.40.65:49965)
0806 22:46:52.669434 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:52.669448 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:52.669522 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:52.669691 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:52.669692 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:52.669694 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:46:52.669785 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:52.669788 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:52.670453 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:52.670454 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:52.671391 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:52.671395 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:52.671913 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:52.671914 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:52.671998 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:52.672003 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:46:52.672056 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 22:46:52.672095 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"mutex_wait_us":47,"thread_start_us":149,"threads_started":1}
I20260806 22:46:52.672307 8551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:52.669398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.65:32861 (local address 127.6.40.66:40507)
0806 22:46:52.669457 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:52.669459 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:52.669572 (+ 113us) server_negotiation.cc:407] Connection header received
0806 22:46:52.669644 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:52.669663 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:52.669679 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:52.669729 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:46:52.669915 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:52.670378 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:52.671405 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:52.671895 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:52.671968 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:52.671980 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:52.672177 (+ 197us) server_negotiation.cc:299] Negotiation successful
0806 22:46:52.672224 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 22:46:52.672420 8566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:52.669281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.66:40507 (local address 127.6.40.65:32861)
0806 22:46:52.669487 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:52.669498 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:52.669570 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:52.669754 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:52.669755 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:52.669757 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:46:52.669818 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:52.669819 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:52.670486 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:52.670487 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:52.671290 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:52.671292 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:52.671993 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:52.671994 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:52.672072 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:52.672077 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:46:52.672136 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 22:46:52.672176 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"mutex_wait_us":18,"thread_start_us":75,"threads_started":1}
I20260806 22:46:52.672617 8462 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c5bbfa118924e11ba1b556c518310e2" candidate_uuid: "9ab7e43baddc4a6aaa3ef480a35e0896" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fab98111c13547c8895666985780a499" is_pre_election: true
I20260806 22:46:52.672775 8462 raft_consensus.cc:2454] T 5c5bbfa118924e11ba1b556c518310e2 P fab98111c13547c8895666985780a499 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9ab7e43baddc4a6aaa3ef480a35e0896 in term 0.
I20260806 22:46:52.672806 8262 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c5bbfa118924e11ba1b556c518310e2" candidate_uuid: "9ab7e43baddc4a6aaa3ef480a35e0896" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "690eb04bf30e4fadac3f1b2ac9125758" is_pre_election: true
I20260806 22:46:52.673028 8262 raft_consensus.cc:2454] T 5c5bbfa118924e11ba1b556c518310e2 P 690eb04bf30e4fadac3f1b2ac9125758 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9ab7e43baddc4a6aaa3ef480a35e0896 in term 0.
I20260806 22:46:52.673085 7976 leader_election.cc:302] T 5c5bbfa118924e11ba1b556c518310e2 P 9ab7e43baddc4a6aaa3ef480a35e0896 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9ab7e43baddc4a6aaa3ef480a35e0896, fab98111c13547c8895666985780a499; no voters:
I20260806 22:46:52.673235 8564 raft_consensus.cc:2781] T 5c5bbfa118924e11ba1b556c518310e2 P 9ab7e43baddc4a6aaa3ef480a35e0896 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:46:52.673295 8564 raft_consensus.cc:482] T 5c5bbfa118924e11ba1b556c518310e2 P 9ab7e43baddc4a6aaa3ef480a35e0896 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:46:52.673321 8564 raft_consensus.cc:3037] T 5c5bbfa118924e11ba1b556c518310e2 P 9ab7e43baddc4a6aaa3ef480a35e0896 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:46:52.674118 8564 raft_consensus.cc:504] T 5c5bbfa118924e11ba1b556c518310e2 P 9ab7e43baddc4a6aaa3ef480a35e0896 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ab7e43baddc4a6aaa3ef480a35e0896" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 43723 } } peers { permanent_uuid: "fab98111c13547c8895666985780a499" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45667 } } peers { permanent_uuid: "690eb04bf30e4fadac3f1b2ac9125758" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 40507 } }
I20260806 22:46:52.674273 8564 leader_election.cc:288] T 5c5bbfa118924e11ba1b556c518310e2 P 9ab7e43baddc4a6aaa3ef480a35e0896 [CANDIDATE]: Term 1 election: Requested vote from peers fab98111c13547c8895666985780a499 (127.6.40.67:45667), 690eb04bf30e4fadac3f1b2ac9125758 (127.6.40.66:40507)
I20260806 22:46:52.674432 8462 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c5bbfa118924e11ba1b556c518310e2" candidate_uuid: "9ab7e43baddc4a6aaa3ef480a35e0896" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fab98111c13547c8895666985780a499"
I20260806 22:46:52.674490 8262 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c5bbfa118924e11ba1b556c518310e2" candidate_uuid: "9ab7e43baddc4a6aaa3ef480a35e0896" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "690eb04bf30e4fadac3f1b2ac9125758"
I20260806 22:46:52.674510 8462 raft_consensus.cc:3037] T 5c5bbfa118924e11ba1b556c518310e2 P fab98111c13547c8895666985780a499 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:46:52.674589 8262 raft_consensus.cc:3037] T 5c5bbfa118924e11ba1b556c518310e2 P 690eb04bf30e4fadac3f1b2ac9125758 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:46:52.675325 8462 raft_consensus.cc:2454] T 5c5bbfa118924e11ba1b556c518310e2 P fab98111c13547c8895666985780a499 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9ab7e43baddc4a6aaa3ef480a35e0896 in term 1.
I20260806 22:46:52.675485 8262 raft_consensus.cc:2454] T 5c5bbfa118924e11ba1b556c518310e2 P 690eb04bf30e4fadac3f1b2ac9125758 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9ab7e43baddc4a6aaa3ef480a35e0896 in term 1.
I20260806 22:46:52.675505 7976 leader_election.cc:302] T 5c5bbfa118924e11ba1b556c518310e2 P 9ab7e43baddc4a6aaa3ef480a35e0896 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9ab7e43baddc4a6aaa3ef480a35e0896, fab98111c13547c8895666985780a499; no voters:
I20260806 22:46:52.675607 8564 raft_consensus.cc:2781] T 5c5bbfa118924e11ba1b556c518310e2 P 9ab7e43baddc4a6aaa3ef480a35e0896 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:46:52.675772 8564 raft_consensus.cc:686] T 5c5bbfa118924e11ba1b556c518310e2 P 9ab7e43baddc4a6aaa3ef480a35e0896 [term 1 LEADER]: Becoming Leader. State: Replica: 9ab7e43baddc4a6aaa3ef480a35e0896, State: Running, Role: LEADER
I20260806 22:46:52.675871 8564 consensus_queue.cc:237] T 5c5bbfa118924e11ba1b556c518310e2 P 9ab7e43baddc4a6aaa3ef480a35e0896 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ab7e43baddc4a6aaa3ef480a35e0896" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 43723 } } peers { permanent_uuid: "fab98111c13547c8895666985780a499" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45667 } } peers { permanent_uuid: "690eb04bf30e4fadac3f1b2ac9125758" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 40507 } }
I20260806 22:46:52.676682 7850 catalog_manager.cc:5495] T 5c5bbfa118924e11ba1b556c518310e2 P 9ab7e43baddc4a6aaa3ef480a35e0896 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ab7e43baddc4a6aaa3ef480a35e0896 (127.6.40.65). New cstate: current_term: 1 leader_uuid: "9ab7e43baddc4a6aaa3ef480a35e0896" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ab7e43baddc4a6aaa3ef480a35e0896" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 43723 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fab98111c13547c8895666985780a499" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "690eb04bf30e4fadac3f1b2ac9125758" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 40507 } health_report { overall_health: UNKNOWN } } }
I20260806 22:46:52.701071 8550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:52.698428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40920 (local address 127.6.40.65:43723)
0806 22:46:52.698506 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:52.698508 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:52.698555 (+ 47us) server_negotiation.cc:407] Connection header received
0806 22:46:52.698644 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:52.698645 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:52.698666 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:52.698730 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:46:52.698884 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:52.699394 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:52.700250 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:52.700452 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:52.700502 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:52.700719 (+ 217us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:46:52.700815 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:46:52.700909 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:52.700987 (+ 78us) server_negotiation.cc:299] Negotiation successful
0806 22:46:52.701008 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 22:46:52.703292 8462 raft_consensus.cc:1261] T 5c5bbfa118924e11ba1b556c518310e2 P fab98111c13547c8895666985780a499 [term 1 FOLLOWER]: Refusing update from remote peer 9ab7e43baddc4a6aaa3ef480a35e0896: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:46:52.703807 8568 consensus_queue.cc:1035] T 5c5bbfa118924e11ba1b556c518310e2 P 9ab7e43baddc4a6aaa3ef480a35e0896 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fab98111c13547c8895666985780a499" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:46:52.705276 8262 raft_consensus.cc:1261] T 5c5bbfa118924e11ba1b556c518310e2 P 690eb04bf30e4fadac3f1b2ac9125758 [term 1 FOLLOWER]: Refusing update from remote peer 9ab7e43baddc4a6aaa3ef480a35e0896: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:46:52.705714 8564 consensus_queue.cc:1035] T 5c5bbfa118924e11ba1b556c518310e2 P 9ab7e43baddc4a6aaa3ef480a35e0896 [LEADER]: Connected to new peer: Peer: permanent_uuid: "690eb04bf30e4fadac3f1b2ac9125758" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 40507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:46:52.706809 8571 mvcc.cc:205] Tried to move back new op lower bound from 7315687066428964864 to 7315687066320961536. Current Snapshot: MvccSnapshot[applied={T|T < 7315687066428964864}]
I20260806 22:46:52.707288 8574 mvcc.cc:205] Tried to move back new op lower bound from 7315687066428964864 to 7315687066320961536. Current Snapshot: MvccSnapshot[applied={T|T < 7315687066428964864}]
I20260806 22:46:52.709219 8577 mvcc.cc:205] Tried to move back new op lower bound from 7315687066428964864 to 7315687066320961536. Current Snapshot: MvccSnapshot[applied={T|T < 7315687066428964864}]
I20260806 22:46:52.712114 7850 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44256:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:46:52.712224 7850 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44256:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:46:52.713078 7850 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 23483819909e4fd5a091b0e031dffb15: Sending DeleteTablet for 3 replicas of tablet 5c5bbfa118924e11ba1b556c518310e2
I20260806 22:46:52.713443 8049 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c5bbfa118924e11ba1b556c518310e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:46:52 UTC) from {username='kudu', principal='kudu/127.6.40.126@KRBTEST.COM'} at 127.0.0.1:40912
I20260806 22:46:52.713457 8431 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c5bbfa118924e11ba1b556c518310e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:46:52 UTC) from {username='kudu', principal='kudu/127.6.40.126@KRBTEST.COM'} at 127.0.0.1:51258
I20260806 22:46:52.713778 8237 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c5bbfa118924e11ba1b556c518310e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:46:52 UTC) from {username='kudu', principal='kudu/127.6.40.126@KRBTEST.COM'} at 127.0.0.1:34104
I20260806 22:46:52.713989 8584 tablet_replica.cc:330] T 5c5bbfa118924e11ba1b556c518310e2 P 690eb04bf30e4fadac3f1b2ac9125758: stopping tablet replica
I20260806 22:46:52.713987 7850 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44256:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 22:46:52.714110 8584 raft_consensus.cc:2229] T 5c5bbfa118924e11ba1b556c518310e2 P 690eb04bf30e4fadac3f1b2ac9125758 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260806 22:46:52.714143 7850 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:46:52.714223 8584 raft_consensus.cc:2258] T 5c5bbfa118924e11ba1b556c518310e2 P 690eb04bf30e4fadac3f1b2ac9125758 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:46:52.714471 8583 tablet_replica.cc:330] T 5c5bbfa118924e11ba1b556c518310e2 P fab98111c13547c8895666985780a499: stopping tablet replica
I20260806 22:46:52.714638 8583 raft_consensus.cc:2229] T 5c5bbfa118924e11ba1b556c518310e2 P fab98111c13547c8895666985780a499 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:46:52.714699 8584 ts_tablet_manager.cc:1905] T 5c5bbfa118924e11ba1b556c518310e2 P 690eb04bf30e4fadac3f1b2ac9125758: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:46:52.714839 8583 raft_consensus.cc:2258] T 5c5bbfa118924e11ba1b556c518310e2 P fab98111c13547c8895666985780a499 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:46:52.715341 8583 ts_tablet_manager.cc:1905] T 5c5bbfa118924e11ba1b556c518310e2 P fab98111c13547c8895666985780a499: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:46:52.715520 8431 tablet_service.cc:1467] Processing CreateTablet for tablet 5c469437677c412fb1468d4f84f5e7c5 (DEFAULT_TABLE table=test-table [id=13eebd36f5294bdd91ddc7fa74aad52e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:46:52.715658 8431 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c469437677c412fb1468d4f84f5e7c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:46:52.716809 8584 ts_tablet_manager.cc:1918] T 5c5bbfa118924e11ba1b556c518310e2 P 690eb04bf30e4fadac3f1b2ac9125758: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:46:52.716918 8584 log.cc:1196] T 5c5bbfa118924e11ba1b556c518310e2 P 690eb04bf30e4fadac3f1b2ac9125758: Deleting WAL directory at /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/minicluster-data/ts-1/wal/wals/5c5bbfa118924e11ba1b556c518310e2
I20260806 22:46:52.717139 8556 tablet_bootstrap.cc:492] T 5c469437677c412fb1468d4f84f5e7c5 P fab98111c13547c8895666985780a499: Bootstrap starting.
I20260806 22:46:52.717196 8584 ts_tablet_manager.cc:1939] T 5c5bbfa118924e11ba1b556c518310e2 P 690eb04bf30e4fadac3f1b2ac9125758: Deleting consensus metadata
I20260806 22:46:52.717494 8049 tablet_service.cc:1467] Processing CreateTablet for tablet 5c469437677c412fb1468d4f84f5e7c5 (DEFAULT_TABLE table=test-table [id=13eebd36f5294bdd91ddc7fa74aad52e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:46:52.717621 8556 tablet_bootstrap.cc:654] T 5c469437677c412fb1468d4f84f5e7c5 P fab98111c13547c8895666985780a499: Neither blocks nor log segments found. Creating new log.
I20260806 22:46:52.717644 8049 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c469437677c412fb1468d4f84f5e7c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:46:52.718045 8583 ts_tablet_manager.cc:1918] T 5c5bbfa118924e11ba1b556c518310e2 P fab98111c13547c8895666985780a499: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:46:52.718127 8583 log.cc:1196] T 5c5bbfa118924e11ba1b556c518310e2 P fab98111c13547c8895666985780a499: Deleting WAL directory at /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/minicluster-data/ts-2/wal/wals/5c5bbfa118924e11ba1b556c518310e2
I20260806 22:46:52.718387 8583 ts_tablet_manager.cc:1939] T 5c5bbfa118924e11ba1b556c518310e2 P fab98111c13547c8895666985780a499: Deleting consensus metadata
I20260806 22:46:52.718426 8556 tablet_bootstrap.cc:492] T 5c469437677c412fb1468d4f84f5e7c5 P fab98111c13547c8895666985780a499: No bootstrap required, opened a new log
I20260806 22:46:52.718477 8556 ts_tablet_manager.cc:1397] T 5c469437677c412fb1468d4f84f5e7c5 P fab98111c13547c8895666985780a499: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 22:46:52.718675 8556 raft_consensus.cc:348] T 5c469437677c412fb1468d4f84f5e7c5 P fab98111c13547c8895666985780a499 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ab7e43baddc4a6aaa3ef480a35e0896" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 43723 } } peers { permanent_uuid: "690eb04bf30e4fadac3f1b2ac9125758" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 40507 } } peers { permanent_uuid: "fab98111c13547c8895666985780a499" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45667 } }
I20260806 22:46:52.718763 8556 raft_consensus.cc:374] T 5c469437677c412fb1468d4f84f5e7c5 P fab98111c13547c8895666985780a499 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:46:52.718784 8556 raft_consensus.cc:729] T 5c469437677c412fb1468d4f84f5e7c5 P fab98111c13547c8895666985780a499 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fab98111c13547c8895666985780a499, State: Initialized, Role: FOLLOWER
I20260806 22:46:52.718829 8556 consensus_queue.cc:260] T 5c469437677c412fb1468d4f84f5e7c5 P fab98111c13547c8895666985780a499 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ab7e43baddc4a6aaa3ef480a35e0896" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 43723 } } peers { permanent_uuid: "690eb04bf30e4fadac3f1b2ac9125758" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 40507 } } peers { permanent_uuid: "fab98111c13547c8895666985780a499" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45667 } }
I20260806 22:46:52.718834 7831 catalog_manager.cc:4841] TS fab98111c13547c8895666985780a499 (127.6.40.67:45667): tablet 5c5bbfa118924e11ba1b556c518310e2 (table test-table [id=2554a0a79f264a40b0c61fd78126a7b2]) successfully deleted
I20260806 22:46:52.718907 8556 ts_tablet_manager.cc:1428] T 5c469437677c412fb1468d4f84f5e7c5 P fab98111c13547c8895666985780a499: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 22:46:52.719328 8557 tablet_bootstrap.cc:492] T 5c469437677c412fb1468d4f84f5e7c5 P 9ab7e43baddc4a6aaa3ef480a35e0896: Bootstrap starting.
I20260806 22:46:52.719748 8557 tablet_bootstrap.cc:654] T 5c469437677c412fb1468d4f84f5e7c5 P 9ab7e43baddc4a6aaa3ef480a35e0896: Neither blocks nor log segments found. Creating new log.
I20260806 22:46:52.720369 7836 catalog_manager.cc:4841] TS 690eb04bf30e4fadac3f1b2ac9125758 (127.6.40.66:40507): tablet 5c5bbfa118924e11ba1b556c518310e2 (table test-table [id=2554a0a79f264a40b0c61fd78126a7b2]) successfully deleted
I20260806 22:46:52.720463 8237 tablet_service.cc:1467] Processing CreateTablet for tablet 5c469437677c412fb1468d4f84f5e7c5 (DEFAULT_TABLE table=test-table [id=13eebd36f5294bdd91ddc7fa74aad52e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:46:52.720566 8237 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c469437677c412fb1468d4f84f5e7c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:46:52.720602 8557 tablet_bootstrap.cc:492] T 5c469437677c412fb1468d4f84f5e7c5 P 9ab7e43baddc4a6aaa3ef480a35e0896: No bootstrap required, opened a new log
I20260806 22:46:52.720646 8557 ts_tablet_manager.cc:1397] T 5c469437677c412fb1468d4f84f5e7c5 P 9ab7e43baddc4a6aaa3ef480a35e0896: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 22:46:52.720799 8557 raft_consensus.cc:348] T 5c469437677c412fb1468d4f84f5e7c5 P 9ab7e43baddc4a6aaa3ef480a35e0896 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ab7e43baddc4a6aaa3ef480a35e0896" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 43723 } } peers { permanent_uuid: "690eb04bf30e4fadac3f1b2ac9125758" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 40507 } } peers { permanent_uuid: "fab98111c13547c8895666985780a499" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45667 } }
I20260806 22:46:52.720902 8557 raft_consensus.cc:374] T 5c469437677c412fb1468d4f84f5e7c5 P 9ab7e43baddc4a6aaa3ef480a35e0896 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:46:52.720927 8557 raft_consensus.cc:729] T 5c469437677c412fb1468d4f84f5e7c5 P 9ab7e43baddc4a6aaa3ef480a35e0896 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ab7e43baddc4a6aaa3ef480a35e0896, State: Initialized, Role: FOLLOWER
I20260806 22:46:52.720975 8557 consensus_queue.cc:260] T 5c469437677c412fb1468d4f84f5e7c5 P 9ab7e43baddc4a6aaa3ef480a35e0896 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ab7e43baddc4a6aaa3ef480a35e0896" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 43723 } } peers { permanent_uuid: "690eb04bf30e4fadac3f1b2ac9125758" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 40507 } } peers { permanent_uuid: "fab98111c13547c8895666985780a499" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45667 } }
I20260806 22:46:52.721062 8557 ts_tablet_manager.cc:1428] T 5c469437677c412fb1468d4f84f5e7c5 P 9ab7e43baddc4a6aaa3ef480a35e0896: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:52.721602 8582 tablet_replica.cc:330] T 5c5bbfa118924e11ba1b556c518310e2 P 9ab7e43baddc4a6aaa3ef480a35e0896: stopping tablet replica
I20260806 22:46:52.721711 8582 raft_consensus.cc:2229] T 5c5bbfa118924e11ba1b556c518310e2 P 9ab7e43baddc4a6aaa3ef480a35e0896 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:46:52.721765 8555 tablet_bootstrap.cc:492] T 5c469437677c412fb1468d4f84f5e7c5 P 690eb04bf30e4fadac3f1b2ac9125758: Bootstrap starting.
I20260806 22:46:52.721830 8582 raft_consensus.cc:2258] T 5c5bbfa118924e11ba1b556c518310e2 P 9ab7e43baddc4a6aaa3ef480a35e0896 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:46:52.722153 8555 tablet_bootstrap.cc:654] T 5c469437677c412fb1468d4f84f5e7c5 P 690eb04bf30e4fadac3f1b2ac9125758: Neither blocks nor log segments found. Creating new log.
I20260806 22:46:52.722275 8582 ts_tablet_manager.cc:1905] T 5c5bbfa118924e11ba1b556c518310e2 P 9ab7e43baddc4a6aaa3ef480a35e0896: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:46:52.723348 8555 tablet_bootstrap.cc:492] T 5c469437677c412fb1468d4f84f5e7c5 P 690eb04bf30e4fadac3f1b2ac9125758: No bootstrap required, opened a new log
I20260806 22:46:52.723428 8555 ts_tablet_manager.cc:1397] T 5c469437677c412fb1468d4f84f5e7c5 P 690eb04bf30e4fadac3f1b2ac9125758: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 22:46:52.723585 8555 raft_consensus.cc:348] T 5c469437677c412fb1468d4f84f5e7c5 P 690eb04bf30e4fadac3f1b2ac9125758 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ab7e43baddc4a6aaa3ef480a35e0896" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 43723 } } peers { permanent_uuid: "690eb04bf30e4fadac3f1b2ac9125758" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 40507 } } peers { permanent_uuid: "fab98111c13547c8895666985780a499" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45667 } }
I20260806 22:46:52.723665 8555 raft_consensus.cc:374] T 5c469437677c412fb1468d4f84f5e7c5 P 690eb04bf30e4fadac3f1b2ac9125758 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:46:52.723693 8555 raft_consensus.cc:729] T 5c469437677c412fb1468d4f84f5e7c5 P 690eb04bf30e4fadac3f1b2ac9125758 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 690eb04bf30e4fadac3f1b2ac9125758, State: Initialized, Role: FOLLOWER
I20260806 22:46:52.723737 8555 consensus_queue.cc:260] T 5c469437677c412fb1468d4f84f5e7c5 P 690eb04bf30e4fadac3f1b2ac9125758 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ab7e43baddc4a6aaa3ef480a35e0896" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 43723 } } peers { permanent_uuid: "690eb04bf30e4fadac3f1b2ac9125758" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 40507 } } peers { permanent_uuid: "fab98111c13547c8895666985780a499" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45667 } }
I20260806 22:46:52.723814 8582 ts_tablet_manager.cc:1918] T 5c5bbfa118924e11ba1b556c518310e2 P 9ab7e43baddc4a6aaa3ef480a35e0896: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:46:52.723839 8555 ts_tablet_manager.cc:1428] T 5c469437677c412fb1468d4f84f5e7c5 P 690eb04bf30e4fadac3f1b2ac9125758: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260806 22:46:52.723897 8582 log.cc:1196] T 5c5bbfa118924e11ba1b556c518310e2 P 9ab7e43baddc4a6aaa3ef480a35e0896: Deleting WAL directory at /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/minicluster-data/ts-0/wal/wals/5c5bbfa118924e11ba1b556c518310e2
I20260806 22:46:52.724156 8582 ts_tablet_manager.cc:1939] T 5c5bbfa118924e11ba1b556c518310e2 P 9ab7e43baddc4a6aaa3ef480a35e0896: Deleting consensus metadata
I20260806 22:46:52.724545 7835 catalog_manager.cc:4841] TS 9ab7e43baddc4a6aaa3ef480a35e0896 (127.6.40.65:43723): tablet 5c5bbfa118924e11ba1b556c518310e2 (table test-table [id=2554a0a79f264a40b0c61fd78126a7b2]) successfully deleted
I20260806 22:46:52.818498 8563 raft_consensus.cc:482] T 5c469437677c412fb1468d4f84f5e7c5 P 690eb04bf30e4fadac3f1b2ac9125758 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:46:52.818634 8563 raft_consensus.cc:504] T 5c469437677c412fb1468d4f84f5e7c5 P 690eb04bf30e4fadac3f1b2ac9125758 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ab7e43baddc4a6aaa3ef480a35e0896" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 43723 } } peers { permanent_uuid: "690eb04bf30e4fadac3f1b2ac9125758" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 40507 } } peers { permanent_uuid: "fab98111c13547c8895666985780a499" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45667 } }
I20260806 22:46:52.819198 8563 leader_election.cc:288] T 5c469437677c412fb1468d4f84f5e7c5 P 690eb04bf30e4fadac3f1b2ac9125758 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9ab7e43baddc4a6aaa3ef480a35e0896 (127.6.40.65:43723), fab98111c13547c8895666985780a499 (127.6.40.67:45667)
I20260806 22:46:52.823215 8550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:52.819507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.66:58379 (local address 127.6.40.65:43723)
0806 22:46:52.819551 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:52.819553 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:52.819564 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:46:52.819640 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:52.819641 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:52.819651 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:52.819706 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:46:52.820055 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:52.820760 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:52.822247 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:52.822812 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:52.822863 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:52.822874 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:52.823104 (+ 230us) server_negotiation.cc:299] Negotiation successful
0806 22:46:52.823139 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 22:46:52.823665 8588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:52.819329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.65:43723 (local address 127.6.40.66:58379)
0806 22:46:52.819477 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:52.819492 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:52.819559 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:52.819885 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:52.819886 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:52.819889 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:46:52.819973 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:52.819975 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:52.820886 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:52.820887 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:52.822130 (+ 1243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:52.822133 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:52.822955 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:52.822956 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:52.823029 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:52.823034 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:46:52.823520 (+ 486us) client_negotiation.cc:240] Negotiation successful
0806 22:46:52.823570 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":71,"threads_started":1}
I20260806 22:46:52.824136 8069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c469437677c412fb1468d4f84f5e7c5" candidate_uuid: "690eb04bf30e4fadac3f1b2ac9125758" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ab7e43baddc4a6aaa3ef480a35e0896" is_pre_election: true
I20260806 22:46:52.824277 8069 raft_consensus.cc:2454] T 5c469437677c412fb1468d4f84f5e7c5 P 9ab7e43baddc4a6aaa3ef480a35e0896 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 690eb04bf30e4fadac3f1b2ac9125758 in term 0.
I20260806 22:46:52.824575 8172 leader_election.cc:302] T 5c469437677c412fb1468d4f84f5e7c5 P 690eb04bf30e4fadac3f1b2ac9125758 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 690eb04bf30e4fadac3f1b2ac9125758, 9ab7e43baddc4a6aaa3ef480a35e0896; no voters:
I20260806 22:46:52.824699 8553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:52.819786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.66:58813 (local address 127.6.40.67:45667)
0806 22:46:52.819847 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:52.819850 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:52.819864 (+ 14us) server_negotiation.cc:407] Connection header received
0806 22:46:52.819901 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:52.819902 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:52.819910 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:52.819986 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:46:52.820253 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:52.821201 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:52.823212 (+ 2011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:52.824011 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:52.824086 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:52.824098 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:52.824480 (+ 382us) server_negotiation.cc:299] Negotiation successful
0806 22:46:52.824553 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 22:46:52.824764 8563 raft_consensus.cc:2781] T 5c469437677c412fb1468d4f84f5e7c5 P 690eb04bf30e4fadac3f1b2ac9125758 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:46:52.824807 8563 raft_consensus.cc:482] T 5c469437677c412fb1468d4f84f5e7c5 P 690eb04bf30e4fadac3f1b2ac9125758 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:46:52.824826 8563 raft_consensus.cc:3037] T 5c469437677c412fb1468d4f84f5e7c5 P 690eb04bf30e4fadac3f1b2ac9125758 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:46:52.825229 8590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:52.819506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.67:45667 (local address 127.6.40.66:58813)
0806 22:46:52.819728 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:52.819739 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:52.819799 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:52.819990 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:52.819991 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:52.819993 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:46:52.820117 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:52.820119 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:52.821398 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:52.821399 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:52.823010 (+ 1611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:52.823013 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:52.824179 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:52.824180 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:52.824340 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:52.824345 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:46:52.825000 (+ 655us) client_negotiation.cc:240] Negotiation successful
0806 22:46:52.825065 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":56,"threads_started":1}
I20260806 22:46:52.825556 8462 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c469437677c412fb1468d4f84f5e7c5" candidate_uuid: "690eb04bf30e4fadac3f1b2ac9125758" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fab98111c13547c8895666985780a499" is_pre_election: true
I20260806 22:46:52.825685 8462 raft_consensus.cc:2454] T 5c469437677c412fb1468d4f84f5e7c5 P fab98111c13547c8895666985780a499 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 690eb04bf30e4fadac3f1b2ac9125758 in term 0.
I20260806 22:46:52.825958 8563 raft_consensus.cc:504] T 5c469437677c412fb1468d4f84f5e7c5 P 690eb04bf30e4fadac3f1b2ac9125758 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ab7e43baddc4a6aaa3ef480a35e0896" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 43723 } } peers { permanent_uuid: "690eb04bf30e4fadac3f1b2ac9125758" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 40507 } } peers { permanent_uuid: "fab98111c13547c8895666985780a499" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45667 } }
I20260806 22:46:52.826115 8563 leader_election.cc:288] T 5c469437677c412fb1468d4f84f5e7c5 P 690eb04bf30e4fadac3f1b2ac9125758 [CANDIDATE]: Term 1 election: Requested vote from peers 9ab7e43baddc4a6aaa3ef480a35e0896 (127.6.40.65:43723), fab98111c13547c8895666985780a499 (127.6.40.67:45667)
I20260806 22:46:52.826287 8462 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c469437677c412fb1468d4f84f5e7c5" candidate_uuid: "690eb04bf30e4fadac3f1b2ac9125758" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fab98111c13547c8895666985780a499"
I20260806 22:46:52.826370 8462 raft_consensus.cc:3037] T 5c469437677c412fb1468d4f84f5e7c5 P fab98111c13547c8895666985780a499 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:46:52.826953 8069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c469437677c412fb1468d4f84f5e7c5" candidate_uuid: "690eb04bf30e4fadac3f1b2ac9125758" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ab7e43baddc4a6aaa3ef480a35e0896"
I20260806 22:46:52.827082 8069 raft_consensus.cc:3037] T 5c469437677c412fb1468d4f84f5e7c5 P 9ab7e43baddc4a6aaa3ef480a35e0896 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:46:52.827561 8462 raft_consensus.cc:2454] T 5c469437677c412fb1468d4f84f5e7c5 P fab98111c13547c8895666985780a499 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 690eb04bf30e4fadac3f1b2ac9125758 in term 1.
I20260806 22:46:52.827730 8168 leader_election.cc:302] T 5c469437677c412fb1468d4f84f5e7c5 P 690eb04bf30e4fadac3f1b2ac9125758 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 690eb04bf30e4fadac3f1b2ac9125758, fab98111c13547c8895666985780a499; no voters:
I20260806 22:46:52.827822 8563 raft_consensus.cc:2781] T 5c469437677c412fb1468d4f84f5e7c5 P 690eb04bf30e4fadac3f1b2ac9125758 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:46:52.827958 8563 raft_consensus.cc:686] T 5c469437677c412fb1468d4f84f5e7c5 P 690eb04bf30e4fadac3f1b2ac9125758 [term 1 LEADER]: Becoming Leader. State: Replica: 690eb04bf30e4fadac3f1b2ac9125758, State: Running, Role: LEADER
I20260806 22:46:52.828059 8563 consensus_queue.cc:237] T 5c469437677c412fb1468d4f84f5e7c5 P 690eb04bf30e4fadac3f1b2ac9125758 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ab7e43baddc4a6aaa3ef480a35e0896" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 43723 } } peers { permanent_uuid: "690eb04bf30e4fadac3f1b2ac9125758" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 40507 } } peers { permanent_uuid: "fab98111c13547c8895666985780a499" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45667 } }
I20260806 22:46:52.828967 7849 catalog_manager.cc:5495] T 5c469437677c412fb1468d4f84f5e7c5 P 690eb04bf30e4fadac3f1b2ac9125758 reported cstate change: term changed from 0 to 1, leader changed from <none> to 690eb04bf30e4fadac3f1b2ac9125758 (127.6.40.66). New cstate: current_term: 1 leader_uuid: "690eb04bf30e4fadac3f1b2ac9125758" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ab7e43baddc4a6aaa3ef480a35e0896" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 43723 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "690eb04bf30e4fadac3f1b2ac9125758" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 40507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fab98111c13547c8895666985780a499" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45667 } health_report { overall_health: UNKNOWN } } }
I20260806 22:46:52.829062 8069 raft_consensus.cc:2454] T 5c469437677c412fb1468d4f84f5e7c5 P 9ab7e43baddc4a6aaa3ef480a35e0896 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 690eb04bf30e4fadac3f1b2ac9125758 in term 1.
Aug 06 22:46:52 dist-test-slave-zh2d krb5kdc[7792](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056411, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.126@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
I20260806 22:46:52.876744 7924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:52.869640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44260 (local address 127.6.40.126:44775)
0806 22:46:52.869739 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:52.869741 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:52.869973 (+ 232us) server_negotiation.cc:407] Connection header received
0806 22:46:52.870036 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:52.870037 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:52.870046 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:52.870100 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:52.870585 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:52.871168 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:52.871872 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:52.872148 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:52.872220 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:52.874406 (+ 2186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:52.874408 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:52.874408 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:46:52.874415 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:52.875702 (+ 1287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:52.876059 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:52.876059 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:52.876060 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:52.876109 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:52.876276 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:52.876276 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:52.876276 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:52.876371 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:52.876423 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:52.876563 (+ 140us) server_negotiation.cc:299] Negotiation successful
0806 22:46:52.876671 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":44,"server-negotiator.queue_time_us":65}
I20260806 22:46:52.876941 8603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:52.869568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:44775 (local address 127.0.0.1:44260)
0806 22:46:52.869748 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:52.869758 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:52.869974 (+ 216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:52.870119 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:52.870119 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:52.870449 (+ 330us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:46:52.870501 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:52.870504 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:52.871239 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:52.871240 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:52.871777 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:52.871779 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:52.872228 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:52.872229 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:52.872273 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:52.872790 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:46:52.872795 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:46:52.874310 (+ 1515us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:46:52.875776 (+ 1466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:52.875783 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:52.875785 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:52.875993 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:52.876167 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:52.876168 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:52.876168 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:52.876223 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:52.876427 (+ 204us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:46:52.876436 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:46:52.876479 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 22:46:52.876532 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 22:46:52.876623 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":74,"threads_started":1}
I20260806 22:46:52.879225 7849 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.6.40.126@KRBTEST.COM'} at 127.0.0.1:44260:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 22:46:52.879485 7849 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:46:52.881114 8049 tablet_service.cc:1467] Processing CreateTablet for tablet a2ace60b0ded4c1a94f408f3204f0e64 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ac6d6568d75741d8b3f72fb58377c1af]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 22:46:52.881135 8237 tablet_service.cc:1467] Processing CreateTablet for tablet a2ace60b0ded4c1a94f408f3204f0e64 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ac6d6568d75741d8b3f72fb58377c1af]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 22:46:52.881304 8237 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2ace60b0ded4c1a94f408f3204f0e64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:46:52.881306 8049 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2ace60b0ded4c1a94f408f3204f0e64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:46:52.883538 8557 tablet_bootstrap.cc:492] T a2ace60b0ded4c1a94f408f3204f0e64 P 9ab7e43baddc4a6aaa3ef480a35e0896: Bootstrap starting.
I20260806 22:46:52.883553 8555 tablet_bootstrap.cc:492] T a2ace60b0ded4c1a94f408f3204f0e64 P 690eb04bf30e4fadac3f1b2ac9125758: Bootstrap starting.
I20260806 22:46:52.883661 8431 tablet_service.cc:1467] Processing CreateTablet for tablet a2ace60b0ded4c1a94f408f3204f0e64 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ac6d6568d75741d8b3f72fb58377c1af]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 22:46:52.883808 8431 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2ace60b0ded4c1a94f408f3204f0e64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:46:52.884152 8557 tablet_bootstrap.cc:654] T a2ace60b0ded4c1a94f408f3204f0e64 P 9ab7e43baddc4a6aaa3ef480a35e0896: Neither blocks nor log segments found. Creating new log.
I20260806 22:46:52.884615 8555 tablet_bootstrap.cc:654] T a2ace60b0ded4c1a94f408f3204f0e64 P 690eb04bf30e4fadac3f1b2ac9125758: Neither blocks nor log segments found. Creating new log.
I20260806 22:46:52.885231 8557 tablet_bootstrap.cc:492] T a2ace60b0ded4c1a94f408f3204f0e64 P 9ab7e43baddc4a6aaa3ef480a35e0896: No bootstrap required, opened a new log
I20260806 22:46:52.885293 8557 ts_tablet_manager.cc:1397] T a2ace60b0ded4c1a94f408f3204f0e64 P 9ab7e43baddc4a6aaa3ef480a35e0896: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:46:52.885510 8557 raft_consensus.cc:348] T a2ace60b0ded4c1a94f408f3204f0e64 P 9ab7e43baddc4a6aaa3ef480a35e0896 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ab7e43baddc4a6aaa3ef480a35e0896" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 43723 } } peers { permanent_uuid: "690eb04bf30e4fadac3f1b2ac9125758" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 40507 } } peers { permanent_uuid: "fab98111c13547c8895666985780a499" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45667 } }
I20260806 22:46:52.885628 8557 raft_consensus.cc:374] T a2ace60b0ded4c1a94f408f3204f0e64 P 9ab7e43baddc4a6aaa3ef480a35e0896 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:46:52.885633 8556 tablet_bootstrap.cc:492] T a2ace60b0ded4c1a94f408f3204f0e64 P fab98111c13547c8895666985780a499: Bootstrap starting.
I20260806 22:46:52.885655 8557 raft_consensus.cc:729] T a2ace60b0ded4c1a94f408f3204f0e64 P 9ab7e43baddc4a6aaa3ef480a35e0896 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ab7e43baddc4a6aaa3ef480a35e0896, State: Initialized, Role: FOLLOWER
I20260806 22:46:52.885697 8557 consensus_queue.cc:260] T a2ace60b0ded4c1a94f408f3204f0e64 P 9ab7e43baddc4a6aaa3ef480a35e0896 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ab7e43baddc4a6aaa3ef480a35e0896" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 43723 } } peers { permanent_uuid: "690eb04bf30e4fadac3f1b2ac9125758" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 40507 } } peers { permanent_uuid: "fab98111c13547c8895666985780a499" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45667 } }
I20260806 22:46:52.885844 8557 ts_tablet_manager.cc:1428] T a2ace60b0ded4c1a94f408f3204f0e64 P 9ab7e43baddc4a6aaa3ef480a35e0896: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 22:46:52.885869 8564 tablet_replica.cc:439] T a2ace60b0ded4c1a94f408f3204f0e64 P 9ab7e43baddc4a6aaa3ef480a35e0896: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ab7e43baddc4a6aaa3ef480a35e0896" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 43723 } } peers { permanent_uuid: "690eb04bf30e4fadac3f1b2ac9125758" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 40507 } } peers { permanent_uuid: "fab98111c13547c8895666985780a499" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45667 } } }
I20260806 22:46:52.886106 8564 tablet_replica.cc:442] T a2ace60b0ded4c1a94f408f3204f0e64 P 9ab7e43baddc4a6aaa3ef480a35e0896: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:46:52.886210 8555 tablet_bootstrap.cc:492] T a2ace60b0ded4c1a94f408f3204f0e64 P 690eb04bf30e4fadac3f1b2ac9125758: No bootstrap required, opened a new log
I20260806 22:46:52.886246 8555 ts_tablet_manager.cc:1397] T a2ace60b0ded4c1a94f408f3204f0e64 P 690eb04bf30e4fadac3f1b2ac9125758: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260806 22:46:52.886653 8555 raft_consensus.cc:348] T a2ace60b0ded4c1a94f408f3204f0e64 P 690eb04bf30e4fadac3f1b2ac9125758 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ab7e43baddc4a6aaa3ef480a35e0896" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 43723 } } peers { permanent_uuid: "690eb04bf30e4fadac3f1b2ac9125758" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 40507 } } peers { permanent_uuid: "fab98111c13547c8895666985780a499" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45667 } }
I20260806 22:46:52.886724 8556 tablet_bootstrap.cc:654] T a2ace60b0ded4c1a94f408f3204f0e64 P fab98111c13547c8895666985780a499: Neither blocks nor log segments found. Creating new log.
I20260806 22:46:52.886757 8555 raft_consensus.cc:374] T a2ace60b0ded4c1a94f408f3204f0e64 P 690eb04bf30e4fadac3f1b2ac9125758 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:46:52.886798 8555 raft_consensus.cc:729] T a2ace60b0ded4c1a94f408f3204f0e64 P 690eb04bf30e4fadac3f1b2ac9125758 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 690eb04bf30e4fadac3f1b2ac9125758, State: Initialized, Role: FOLLOWER
I20260806 22:46:52.886847 8555 consensus_queue.cc:260] T a2ace60b0ded4c1a94f408f3204f0e64 P 690eb04bf30e4fadac3f1b2ac9125758 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ab7e43baddc4a6aaa3ef480a35e0896" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 43723 } } peers { permanent_uuid: "690eb04bf30e4fadac3f1b2ac9125758" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 40507 } } peers { permanent_uuid: "fab98111c13547c8895666985780a499" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45667 } }
I20260806 22:46:52.886958 8555 ts_tablet_manager.cc:1428] T a2ace60b0ded4c1a94f408f3204f0e64 P 690eb04bf30e4fadac3f1b2ac9125758: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 22:46:52.887110 8592 tablet_replica.cc:439] T a2ace60b0ded4c1a94f408f3204f0e64 P 690eb04bf30e4fadac3f1b2ac9125758: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ab7e43baddc4a6aaa3ef480a35e0896" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 43723 } } peers { permanent_uuid: "690eb04bf30e4fadac3f1b2ac9125758" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 40507 } } peers { permanent_uuid: "fab98111c13547c8895666985780a499" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45667 } } }
I20260806 22:46:52.887207 8592 tablet_replica.cc:442] T a2ace60b0ded4c1a94f408f3204f0e64 P 690eb04bf30e4fadac3f1b2ac9125758: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:46:52.888065 8556 tablet_bootstrap.cc:492] T a2ace60b0ded4c1a94f408f3204f0e64 P fab98111c13547c8895666985780a499: No bootstrap required, opened a new log
I20260806 22:46:52.888118 8556 ts_tablet_manager.cc:1397] T a2ace60b0ded4c1a94f408f3204f0e64 P fab98111c13547c8895666985780a499: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 22:46:52.888448 8556 raft_consensus.cc:348] T a2ace60b0ded4c1a94f408f3204f0e64 P fab98111c13547c8895666985780a499 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ab7e43baddc4a6aaa3ef480a35e0896" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 43723 } } peers { permanent_uuid: "690eb04bf30e4fadac3f1b2ac9125758" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 40507 } } peers { permanent_uuid: "fab98111c13547c8895666985780a499" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45667 } }
I20260806 22:46:52.888514 8556 raft_consensus.cc:374] T a2ace60b0ded4c1a94f408f3204f0e64 P fab98111c13547c8895666985780a499 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:46:52.888528 8556 raft_consensus.cc:729] T a2ace60b0ded4c1a94f408f3204f0e64 P fab98111c13547c8895666985780a499 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fab98111c13547c8895666985780a499, State: Initialized, Role: FOLLOWER
I20260806 22:46:52.888595 8556 consensus_queue.cc:260] T a2ace60b0ded4c1a94f408f3204f0e64 P fab98111c13547c8895666985780a499 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ab7e43baddc4a6aaa3ef480a35e0896" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 43723 } } peers { permanent_uuid: "690eb04bf30e4fadac3f1b2ac9125758" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 40507 } } peers { permanent_uuid: "fab98111c13547c8895666985780a499" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45667 } }
I20260806 22:46:52.888763 8556 ts_tablet_manager.cc:1428] T a2ace60b0ded4c1a94f408f3204f0e64 P fab98111c13547c8895666985780a499: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 22:46:52.888777 8565 tablet_replica.cc:439] T a2ace60b0ded4c1a94f408f3204f0e64 P fab98111c13547c8895666985780a499: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ab7e43baddc4a6aaa3ef480a35e0896" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 43723 } } peers { permanent_uuid: "690eb04bf30e4fadac3f1b2ac9125758" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 40507 } } peers { permanent_uuid: "fab98111c13547c8895666985780a499" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45667 } } }
I20260806 22:46:52.888883 8565 tablet_replica.cc:442] T a2ace60b0ded4c1a94f408f3204f0e64 P fab98111c13547c8895666985780a499: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:46:53.187876 8564 raft_consensus.cc:482] T a2ace60b0ded4c1a94f408f3204f0e64 P 9ab7e43baddc4a6aaa3ef480a35e0896 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:46:53.188024 8564 raft_consensus.cc:504] T a2ace60b0ded4c1a94f408f3204f0e64 P 9ab7e43baddc4a6aaa3ef480a35e0896 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ab7e43baddc4a6aaa3ef480a35e0896" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 43723 } } peers { permanent_uuid: "690eb04bf30e4fadac3f1b2ac9125758" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 40507 } } peers { permanent_uuid: "fab98111c13547c8895666985780a499" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45667 } }
I20260806 22:46:53.188246 8564 leader_election.cc:288] T a2ace60b0ded4c1a94f408f3204f0e64 P 9ab7e43baddc4a6aaa3ef480a35e0896 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 690eb04bf30e4fadac3f1b2ac9125758 (127.6.40.66:40507), fab98111c13547c8895666985780a499 (127.6.40.67:45667)
I20260806 22:46:53.188540 8262 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2ace60b0ded4c1a94f408f3204f0e64" candidate_uuid: "9ab7e43baddc4a6aaa3ef480a35e0896" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "690eb04bf30e4fadac3f1b2ac9125758" is_pre_election: true
I20260806 22:46:53.188550 8462 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2ace60b0ded4c1a94f408f3204f0e64" candidate_uuid: "9ab7e43baddc4a6aaa3ef480a35e0896" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fab98111c13547c8895666985780a499" is_pre_election: true
I20260806 22:46:53.188670 8462 raft_consensus.cc:2454] T a2ace60b0ded4c1a94f408f3204f0e64 P fab98111c13547c8895666985780a499 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9ab7e43baddc4a6aaa3ef480a35e0896 in term 0.
I20260806 22:46:53.188675 8262 raft_consensus.cc:2454] T a2ace60b0ded4c1a94f408f3204f0e64 P 690eb04bf30e4fadac3f1b2ac9125758 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9ab7e43baddc4a6aaa3ef480a35e0896 in term 0.
I20260806 22:46:53.188858 7976 leader_election.cc:302] T a2ace60b0ded4c1a94f408f3204f0e64 P 9ab7e43baddc4a6aaa3ef480a35e0896 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9ab7e43baddc4a6aaa3ef480a35e0896, fab98111c13547c8895666985780a499; no voters:
I20260806 22:46:53.188987 8564 raft_consensus.cc:2781] T a2ace60b0ded4c1a94f408f3204f0e64 P 9ab7e43baddc4a6aaa3ef480a35e0896 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:46:53.189049 8564 raft_consensus.cc:482] T a2ace60b0ded4c1a94f408f3204f0e64 P 9ab7e43baddc4a6aaa3ef480a35e0896 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:46:53.189082 8564 raft_consensus.cc:3037] T a2ace60b0ded4c1a94f408f3204f0e64 P 9ab7e43baddc4a6aaa3ef480a35e0896 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:46:53.190013 8564 raft_consensus.cc:504] T a2ace60b0ded4c1a94f408f3204f0e64 P 9ab7e43baddc4a6aaa3ef480a35e0896 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ab7e43baddc4a6aaa3ef480a35e0896" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 43723 } } peers { permanent_uuid: "690eb04bf30e4fadac3f1b2ac9125758" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 40507 } } peers { permanent_uuid: "fab98111c13547c8895666985780a499" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45667 } }
I20260806 22:46:53.190168 8564 leader_election.cc:288] T a2ace60b0ded4c1a94f408f3204f0e64 P 9ab7e43baddc4a6aaa3ef480a35e0896 [CANDIDATE]: Term 1 election: Requested vote from peers 690eb04bf30e4fadac3f1b2ac9125758 (127.6.40.66:40507), fab98111c13547c8895666985780a499 (127.6.40.67:45667)
I20260806 22:46:53.190363 8462 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2ace60b0ded4c1a94f408f3204f0e64" candidate_uuid: "9ab7e43baddc4a6aaa3ef480a35e0896" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fab98111c13547c8895666985780a499"
I20260806 22:46:53.190466 8462 raft_consensus.cc:3037] T a2ace60b0ded4c1a94f408f3204f0e64 P fab98111c13547c8895666985780a499 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:46:53.190431 8262 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2ace60b0ded4c1a94f408f3204f0e64" candidate_uuid: "9ab7e43baddc4a6aaa3ef480a35e0896" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "690eb04bf30e4fadac3f1b2ac9125758"
I20260806 22:46:53.190538 8262 raft_consensus.cc:3037] T a2ace60b0ded4c1a94f408f3204f0e64 P 690eb04bf30e4fadac3f1b2ac9125758 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:46:53.191210 8462 raft_consensus.cc:2454] T a2ace60b0ded4c1a94f408f3204f0e64 P fab98111c13547c8895666985780a499 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9ab7e43baddc4a6aaa3ef480a35e0896 in term 1.
I20260806 22:46:53.191388 7976 leader_election.cc:302] T a2ace60b0ded4c1a94f408f3204f0e64 P 9ab7e43baddc4a6aaa3ef480a35e0896 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9ab7e43baddc4a6aaa3ef480a35e0896, fab98111c13547c8895666985780a499; no voters:
I20260806 22:46:53.191494 8564 raft_consensus.cc:2781] T a2ace60b0ded4c1a94f408f3204f0e64 P 9ab7e43baddc4a6aaa3ef480a35e0896 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:46:53.191535 8262 raft_consensus.cc:2454] T a2ace60b0ded4c1a94f408f3204f0e64 P 690eb04bf30e4fadac3f1b2ac9125758 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9ab7e43baddc4a6aaa3ef480a35e0896 in term 1.
I20260806 22:46:53.191565 8564 raft_consensus.cc:686] T a2ace60b0ded4c1a94f408f3204f0e64 P 9ab7e43baddc4a6aaa3ef480a35e0896 [term 1 LEADER]: Becoming Leader. State: Replica: 9ab7e43baddc4a6aaa3ef480a35e0896, State: Running, Role: LEADER
I20260806 22:46:53.191613 8564 consensus_queue.cc:237] T a2ace60b0ded4c1a94f408f3204f0e64 P 9ab7e43baddc4a6aaa3ef480a35e0896 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ab7e43baddc4a6aaa3ef480a35e0896" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 43723 } } peers { permanent_uuid: "690eb04bf30e4fadac3f1b2ac9125758" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 40507 } } peers { permanent_uuid: "fab98111c13547c8895666985780a499" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45667 } }
I20260806 22:46:53.191917 8568 tablet_replica.cc:439] T a2ace60b0ded4c1a94f408f3204f0e64 P 9ab7e43baddc4a6aaa3ef480a35e0896: TxnStatusTablet state changed. Reason: New leader 9ab7e43baddc4a6aaa3ef480a35e0896. Latest consensus state: current_term: 1 leader_uuid: "9ab7e43baddc4a6aaa3ef480a35e0896" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ab7e43baddc4a6aaa3ef480a35e0896" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 43723 } } peers { permanent_uuid: "690eb04bf30e4fadac3f1b2ac9125758" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 40507 } } peers { permanent_uuid: "fab98111c13547c8895666985780a499" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45667 } } }
I20260806 22:46:53.192015 8568 tablet_replica.cc:442] T a2ace60b0ded4c1a94f408f3204f0e64 P 9ab7e43baddc4a6aaa3ef480a35e0896: This TxnStatusTablet replica's current role is: LEADER
I20260806 22:46:53.192164 8613 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 22:46:53.192305 8613 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 22:46:53.192330 7850 catalog_manager.cc:5495] T a2ace60b0ded4c1a94f408f3204f0e64 P 9ab7e43baddc4a6aaa3ef480a35e0896 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ab7e43baddc4a6aaa3ef480a35e0896 (127.6.40.65). New cstate: current_term: 1 leader_uuid: "9ab7e43baddc4a6aaa3ef480a35e0896" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ab7e43baddc4a6aaa3ef480a35e0896" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 43723 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "690eb04bf30e4fadac3f1b2ac9125758" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 40507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fab98111c13547c8895666985780a499" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45667 } health_report { overall_health: UNKNOWN } } }
I20260806 22:46:53.237387 8592 consensus_queue.cc:1035] T 5c469437677c412fb1468d4f84f5e7c5 P 690eb04bf30e4fadac3f1b2ac9125758 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fab98111c13547c8895666985780a499" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:46:53.240348 8563 consensus_queue.cc:1035] T 5c469437677c412fb1468d4f84f5e7c5 P 690eb04bf30e4fadac3f1b2ac9125758 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9ab7e43baddc4a6aaa3ef480a35e0896" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 43723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:46:53.241842 8550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:53.238678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40922 (local address 127.6.40.65:43723)
0806 22:46:53.238750 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:53.238753 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:53.238764 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:46:53.238793 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:53.238794 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:53.238806 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:53.238914 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:46:53.239058 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:53.239832 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:53.240958 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:53.241275 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:53.241377 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:53.241565 (+ 188us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:46:53.241606 (+ 41us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:46:53.241652 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:53.241739 (+ 87us) server_negotiation.cc:299] Negotiation successful
0806 22:46:53.241763 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 22:46:53.241856 8603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:53.238362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.65:43723 (local address 127.0.0.1:40922)
0806 22:46:53.238528 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:53.238540 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:53.238596 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:53.238917 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:53.238918 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:53.238920 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 22:46:53.238982 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:53.238985 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:53.239909 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:53.239910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:53.240777 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:53.240780 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:53.241380 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:53.241381 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:53.241442 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:53.241464 (+ 22us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 22:46:53.241678 (+ 214us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 22:46:53.241688 (+ 10us) client_negotiation.cc:769] Sending connection context
0806 22:46:53.241736 (+ 48us) client_negotiation.cc:240] Negotiation successful
0806 22:46:53.241787 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128}
I20260806 22:46:53.242839 8462 raft_consensus.cc:1261] T a2ace60b0ded4c1a94f408f3204f0e64 P fab98111c13547c8895666985780a499 [term 1 FOLLOWER]: Refusing update from remote peer 9ab7e43baddc4a6aaa3ef480a35e0896: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:46:53.242893 8262 raft_consensus.cc:1261] T a2ace60b0ded4c1a94f408f3204f0e64 P 690eb04bf30e4fadac3f1b2ac9125758 [term 1 FOLLOWER]: Refusing update from remote peer 9ab7e43baddc4a6aaa3ef480a35e0896: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:46:53.243077 8568 consensus_queue.cc:1035] T a2ace60b0ded4c1a94f408f3204f0e64 P 9ab7e43baddc4a6aaa3ef480a35e0896 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fab98111c13547c8895666985780a499" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:46:53.243166 8564 consensus_queue.cc:1035] T a2ace60b0ded4c1a94f408f3204f0e64 P 9ab7e43baddc4a6aaa3ef480a35e0896 [LEADER]: Connected to new peer: Peer: permanent_uuid: "690eb04bf30e4fadac3f1b2ac9125758" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 40507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:46:53.243734 8563 tablet_replica.cc:439] T a2ace60b0ded4c1a94f408f3204f0e64 P 690eb04bf30e4fadac3f1b2ac9125758: TxnStatusTablet state changed. Reason: New leader 9ab7e43baddc4a6aaa3ef480a35e0896. Latest consensus state: current_term: 1 leader_uuid: "9ab7e43baddc4a6aaa3ef480a35e0896" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ab7e43baddc4a6aaa3ef480a35e0896" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 43723 } } peers { permanent_uuid: "690eb04bf30e4fadac3f1b2ac9125758" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 40507 } } peers { permanent_uuid: "fab98111c13547c8895666985780a499" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45667 } } }
I20260806 22:46:53.243824 8563 tablet_replica.cc:442] T a2ace60b0ded4c1a94f408f3204f0e64 P 690eb04bf30e4fadac3f1b2ac9125758: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:46:53.244027 8565 tablet_replica.cc:439] T a2ace60b0ded4c1a94f408f3204f0e64 P fab98111c13547c8895666985780a499: TxnStatusTablet state changed. Reason: New leader 9ab7e43baddc4a6aaa3ef480a35e0896. Latest consensus state: current_term: 1 leader_uuid: "9ab7e43baddc4a6aaa3ef480a35e0896" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ab7e43baddc4a6aaa3ef480a35e0896" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 43723 } } peers { permanent_uuid: "690eb04bf30e4fadac3f1b2ac9125758" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 40507 } } peers { permanent_uuid: "fab98111c13547c8895666985780a499" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45667 } } }
I20260806 22:46:53.244117 8565 tablet_replica.cc:442] T a2ace60b0ded4c1a94f408f3204f0e64 P fab98111c13547c8895666985780a499: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:46:53.244480 8568 tablet_replica.cc:439] T a2ace60b0ded4c1a94f408f3204f0e64 P 9ab7e43baddc4a6aaa3ef480a35e0896: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9ab7e43baddc4a6aaa3ef480a35e0896" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ab7e43baddc4a6aaa3ef480a35e0896" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 43723 } } peers { permanent_uuid: "690eb04bf30e4fadac3f1b2ac9125758" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 40507 } } peers { permanent_uuid: "fab98111c13547c8895666985780a499" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45667 } } }
I20260806 22:46:53.244582 8568 tablet_replica.cc:442] T a2ace60b0ded4c1a94f408f3204f0e64 P 9ab7e43baddc4a6aaa3ef480a35e0896: This TxnStatusTablet replica's current role is: LEADER
I20260806 22:46:53.244590 8565 tablet_replica.cc:439] T a2ace60b0ded4c1a94f408f3204f0e64 P fab98111c13547c8895666985780a499: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9ab7e43baddc4a6aaa3ef480a35e0896" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ab7e43baddc4a6aaa3ef480a35e0896" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 43723 } } peers { permanent_uuid: "690eb04bf30e4fadac3f1b2ac9125758" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 40507 } } peers { permanent_uuid: "fab98111c13547c8895666985780a499" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45667 } } }
I20260806 22:46:53.244663 8565 tablet_replica.cc:442] T a2ace60b0ded4c1a94f408f3204f0e64 P fab98111c13547c8895666985780a499: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:46:53.245821 8564 tablet_replica.cc:439] T a2ace60b0ded4c1a94f408f3204f0e64 P 9ab7e43baddc4a6aaa3ef480a35e0896: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9ab7e43baddc4a6aaa3ef480a35e0896" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ab7e43baddc4a6aaa3ef480a35e0896" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 43723 } } peers { permanent_uuid: "690eb04bf30e4fadac3f1b2ac9125758" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 40507 } } peers { permanent_uuid: "fab98111c13547c8895666985780a499" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45667 } } }
I20260806 22:46:53.245904 8564 tablet_replica.cc:442] T a2ace60b0ded4c1a94f408f3204f0e64 P 9ab7e43baddc4a6aaa3ef480a35e0896: This TxnStatusTablet replica's current role is: LEADER
I20260806 22:46:53.246309 8563 tablet_replica.cc:439] T a2ace60b0ded4c1a94f408f3204f0e64 P 690eb04bf30e4fadac3f1b2ac9125758: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9ab7e43baddc4a6aaa3ef480a35e0896" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ab7e43baddc4a6aaa3ef480a35e0896" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 43723 } } peers { permanent_uuid: "690eb04bf30e4fadac3f1b2ac9125758" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 40507 } } peers { permanent_uuid: "fab98111c13547c8895666985780a499" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45667 } } }
I20260806 22:46:53.246449 8563 tablet_replica.cc:442] T a2ace60b0ded4c1a94f408f3204f0e64 P 690eb04bf30e4fadac3f1b2ac9125758: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:46:53.254244 8630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:53.251310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34106 (local address 127.6.40.66:40507)
0806 22:46:53.251438 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:53.251440 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:53.251447 (+ 7us) server_negotiation.cc:407] Connection header received
0806 22:46:53.251534 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:53.251535 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:53.251556 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:53.251656 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:46:53.251822 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:53.252320 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:53.253249 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:53.253477 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:53.253563 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:53.253752 (+ 189us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:46:53.253878 (+ 126us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:46:53.253969 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:53.254026 (+ 57us) server_negotiation.cc:299] Negotiation successful
0806 22:46:53.254051 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":65,"threads_started":1}
I20260806 22:46:53.260370 8550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:53.257378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.66:58651 (local address 127.6.40.65:43723)
0806 22:46:53.257427 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:53.257429 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:53.257694 (+ 265us) server_negotiation.cc:407] Connection header received
0806 22:46:53.257758 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:53.257759 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:53.257769 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:53.257808 (+ 39us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:46:53.257960 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:53.258393 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:53.259577 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:53.259767 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:53.259844 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:53.260087 (+ 243us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:46:53.260120 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:46:53.260182 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:53.260276 (+ 94us) server_negotiation.cc:299] Negotiation successful
0806 22:46:53.260298 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 22:46:53.260385 8635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:53.257263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.65:43723 (local address 127.6.40.66:58651)
0806 22:46:53.257612 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:53.257625 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:53.257692 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:53.257834 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:53.257835 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:53.257837 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 22:46:53.257887 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:53.257888 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:53.258482 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:53.258484 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:53.259470 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:53.259474 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:53.259871 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:53.259873 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:53.259947 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:53.259964 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 22:46:53.260204 (+ 240us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 22:46:53.260207 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:46:53.260261 (+ 54us) client_negotiation.cc:240] Negotiation successful
0806 22:46:53.260309 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":179,"threads_started":1}
I20260806 22:46:53.272527 8630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:53.269242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.65:46193 (local address 127.6.40.66:40507)
0806 22:46:53.269332 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:53.269334 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:53.269586 (+ 252us) server_negotiation.cc:407] Connection header received
0806 22:46:53.269692 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:53.269693 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:53.269702 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:53.269744 (+ 42us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:46:53.269864 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:53.270357 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:53.271595 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:53.271936 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:53.272046 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:53.272254 (+ 208us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:46:53.272292 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:46:53.272349 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:53.272430 (+ 81us) server_negotiation.cc:299] Negotiation successful
0806 22:46:53.272450 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 22:46:53.272547 8639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:53.269170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.66:40507 (local address 127.6.40.65:46193)
0806 22:46:53.269510 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:53.269523 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:53.269615 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:53.269748 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:53.269749 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:53.269750 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 22:46:53.269802 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:53.269804 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:53.270444 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:53.270445 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:53.271479 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:53.271482 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:53.272063 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:53.272064 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:53.272134 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:53.272160 (+ 26us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 22:46:53.272370 (+ 210us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 22:46:53.272372 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 22:46:53.272415 (+ 43us) client_negotiation.cc:240] Negotiation successful
0806 22:46:53.272464 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":179,"threads_started":1}
I20260806 22:46:53.282585 7849 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44256:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:46:53.282698 7849 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44256:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:46:53.283412 7849 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 23483819909e4fd5a091b0e031dffb15: Sending DeleteTablet for 3 replicas of tablet 5c469437677c412fb1468d4f84f5e7c5
I20260806 22:46:53.283667 8237 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c469437677c412fb1468d4f84f5e7c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:46:53 UTC) from {username='kudu', principal='kudu/127.6.40.126@KRBTEST.COM'} at 127.0.0.1:34104
I20260806 22:46:53.283913 8049 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c469437677c412fb1468d4f84f5e7c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:46:53 UTC) from {username='kudu', principal='kudu/127.6.40.126@KRBTEST.COM'} at 127.0.0.1:40912
I20260806 22:46:53.284173 8642 tablet_replica.cc:330] T 5c469437677c412fb1468d4f84f5e7c5 P 690eb04bf30e4fadac3f1b2ac9125758: stopping tablet replica
I20260806 22:46:53.284291 8642 raft_consensus.cc:2229] T 5c469437677c412fb1468d4f84f5e7c5 P 690eb04bf30e4fadac3f1b2ac9125758 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:46:53.284369 8431 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c469437677c412fb1468d4f84f5e7c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:46:53 UTC) from {username='kudu', principal='kudu/127.6.40.126@KRBTEST.COM'} at 127.0.0.1:51258
I20260806 22:46:53.284428 8642 raft_consensus.cc:2258] T 5c469437677c412fb1468d4f84f5e7c5 P 690eb04bf30e4fadac3f1b2ac9125758 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:46:53.284966 8642 ts_tablet_manager.cc:1905] T 5c469437677c412fb1468d4f84f5e7c5 P 690eb04bf30e4fadac3f1b2ac9125758: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:46:53.285418 8646 tablet_replica.cc:330] T 5c469437677c412fb1468d4f84f5e7c5 P fab98111c13547c8895666985780a499: stopping tablet replica
I20260806 22:46:53.285492 8646 raft_consensus.cc:2229] T 5c469437677c412fb1468d4f84f5e7c5 P fab98111c13547c8895666985780a499 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:46:53.285533 8646 raft_consensus.cc:2258] T 5c469437677c412fb1468d4f84f5e7c5 P fab98111c13547c8895666985780a499 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:46:53.286330 8646 ts_tablet_manager.cc:1905] T 5c469437677c412fb1468d4f84f5e7c5 P fab98111c13547c8895666985780a499: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:46:53.288538 8642 ts_tablet_manager.cc:1918] T 5c469437677c412fb1468d4f84f5e7c5 P 690eb04bf30e4fadac3f1b2ac9125758: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:46:53.288655 8642 log.cc:1196] T 5c469437677c412fb1468d4f84f5e7c5 P 690eb04bf30e4fadac3f1b2ac9125758: Deleting WAL directory at /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/minicluster-data/ts-1/wal/wals/5c469437677c412fb1468d4f84f5e7c5
I20260806 22:46:53.289008 8642 ts_tablet_manager.cc:1939] T 5c469437677c412fb1468d4f84f5e7c5 P 690eb04bf30e4fadac3f1b2ac9125758: Deleting consensus metadata
I20260806 22:46:53.289206 8646 ts_tablet_manager.cc:1918] T 5c469437677c412fb1468d4f84f5e7c5 P fab98111c13547c8895666985780a499: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:46:53.289297 8646 log.cc:1196] T 5c469437677c412fb1468d4f84f5e7c5 P fab98111c13547c8895666985780a499: Deleting WAL directory at /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/minicluster-data/ts-2/wal/wals/5c469437677c412fb1468d4f84f5e7c5
I20260806 22:46:53.289533 7836 catalog_manager.cc:4841] TS 690eb04bf30e4fadac3f1b2ac9125758 (127.6.40.66:40507): tablet 5c469437677c412fb1468d4f84f5e7c5 (table test-table [id=13eebd36f5294bdd91ddc7fa74aad52e]) successfully deleted
I20260806 22:46:53.289580 8646 ts_tablet_manager.cc:1939] T 5c469437677c412fb1468d4f84f5e7c5 P fab98111c13547c8895666985780a499: Deleting consensus metadata
I20260806 22:46:53.289868 8644 tablet_replica.cc:330] T 5c469437677c412fb1468d4f84f5e7c5 P 9ab7e43baddc4a6aaa3ef480a35e0896: stopping tablet replica
I20260806 22:46:53.289937 8644 raft_consensus.cc:2229] T 5c469437677c412fb1468d4f84f5e7c5 P 9ab7e43baddc4a6aaa3ef480a35e0896 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:46:53.289923 7831 catalog_manager.cc:4841] TS fab98111c13547c8895666985780a499 (127.6.40.67:45667): tablet 5c469437677c412fb1468d4f84f5e7c5 (table test-table [id=13eebd36f5294bdd91ddc7fa74aad52e]) successfully deleted
I20260806 22:46:53.290004 8644 raft_consensus.cc:2258] T 5c469437677c412fb1468d4f84f5e7c5 P 9ab7e43baddc4a6aaa3ef480a35e0896 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:46:53.290308 8644 ts_tablet_manager.cc:1905] T 5c469437677c412fb1468d4f84f5e7c5 P 9ab7e43baddc4a6aaa3ef480a35e0896: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:46:53.292367 8644 ts_tablet_manager.cc:1918] T 5c469437677c412fb1468d4f84f5e7c5 P 9ab7e43baddc4a6aaa3ef480a35e0896: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:46:53.292441 8644 log.cc:1196] T 5c469437677c412fb1468d4f84f5e7c5 P 9ab7e43baddc4a6aaa3ef480a35e0896: Deleting WAL directory at /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056405724609-6305-0/minicluster-data/ts-0/wal/wals/5c469437677c412fb1468d4f84f5e7c5
I20260806 22:46:53.292717 8644 ts_tablet_manager.cc:1939] T 5c469437677c412fb1468d4f84f5e7c5 P 9ab7e43baddc4a6aaa3ef480a35e0896: Deleting consensus metadata
I20260806 22:46:53.293121 7835 catalog_manager.cc:4841] TS 9ab7e43baddc4a6aaa3ef480a35e0896 (127.6.40.65:43723): tablet 5c469437677c412fb1468d4f84f5e7c5 (table test-table [id=13eebd36f5294bdd91ddc7fa74aad52e]) successfully deleted
Aug 06 22:46:53 dist-test-slave-zh2d krb5kdc[7792](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056412, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.6.40.65@KRBTEST.COM
I20260806 22:46:53.304443 8550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:53.285872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40932 (local address 127.6.40.65:43723)
0806 22:46:53.285911 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:53.285913 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:53.293037 (+ 7124us) server_negotiation.cc:407] Connection header received
0806 22:46:53.293164 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:53.293166 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:53.293177 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:53.293207 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:53.294053 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:53.294870 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:53.295884 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:53.296212 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:53.296290 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:53.300440 (+ 4150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:53.300452 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:53.300453 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:46:53.300461 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:53.302610 (+ 2149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:53.303137 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:53.303141 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:53.303142 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:53.303232 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:53.303529 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:53.303529 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:53.303530 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:53.303706 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:53.303885 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:53.304092 (+ 207us) server_negotiation.cc:299] Negotiation successful
0806 22:46:53.304186 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
W20260806 22:46:53.304770 8103 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40932
I20260806 22:46:53.314944 7924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:53.306048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44274 (local address 127.6.40.126:44775)
0806 22:46:53.306126 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:53.306128 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:53.306461 (+ 333us) server_negotiation.cc:407] Connection header received
0806 22:46:53.306515 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:53.306516 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:53.306530 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:53.306594 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:53.307355 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:53.308153 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:53.309286 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:53.309665 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:53.309749 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:53.311586 (+ 1837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:53.311588 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:53.311588 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:46:53.311597 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:53.313554 (+ 1957us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:53.314025 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:53.314026 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:53.314026 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:53.314096 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:53.314335 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:53.314336 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:53.314336 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:53.314485 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:53.314594 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:53.314747 (+ 153us) server_negotiation.cc:299] Negotiation successful
0806 22:46:53.314835 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
W20260806 22:46:53.315433 7849 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44274
I20260806 22:46:53.316190 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 7950
I20260806 22:46:53.326892 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 8150
I20260806 22:46:53.337178 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 8335
I20260806 22:46:53.344542 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 7812
2026-08-06T22:46:53Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2953 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:46:53 dist-test-slave-zh2d krb5kdc[8654](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:46:53 dist-test-slave-zh2d krb5kdc[8654](info): set up 2 sockets
Aug 06 22:46:53 dist-test-slave-zh2d krb5kdc[8654](info): commencing operation
krb5kdc: starting...
W20260806 22:46:54.566323 6305 mini_kdc.cc:121] Time spent starting KDC: real 1.195s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/krb5kdc/test-user.keytab.
Aug 06 22:46:54 dist-test-slave-zh2d krb5kdc[8654](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056414, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:46:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:46:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.40.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.40.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.40.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab.
Entry for principal kudu/127.6.40.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab.
I20260806 22:46:54.691759 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.126:39721
--webserver_interface=127.6.40.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab
--principal=kudu/127.6.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:36477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.40.126:39721
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:46:54.776744 8669 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:46:54.777043 8669 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:46:54.777088 8669 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:46:54.778396 8669 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:46:54.778471 8669 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 22:46:54.778496 8669 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:46:54.778522 8669 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:46:54.778553 8669 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:46:54.778580 8669 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:46:54.780237 8669 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:36477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.40.126:39721
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab
--principal=kudu/127.6.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.126:39721
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8669
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:46:54.780536 8669 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:46:54.780798 8669 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:46:54.786415 8683 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:54.786425 8680 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:54.786455 8679 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:46:54.786482 8669 server_base.cc:1034] running on GCE node
I20260806 22:46:54.787014 8669 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:46:54.787281 8669 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:46:54.788446 8669 hybrid_clock.cc:648] HybridClock initialized: now 1786056414788425 us; error 38 us; skew 500 ppm
Aug 06 22:46:54 dist-test-slave-zh2d krb5kdc[8654](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056414, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:46:54.791121 8669 init.cc:377] Logged in from keytab as kudu/127.6.40.126@KRBTEST.COM (short username kudu)
I20260806 22:46:54.791568 8669 webserver.cc:460] Webserver started at http://127.6.40.126:43561/ using document root <none> and password file <none>
I20260806 22:46:54.791775 8669 fs_manager.cc:359] Metadata directory not provided
I20260806 22:46:54.791816 8669 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:46:54.791922 8669 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:46:54.793079 8669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/minicluster-data/master-0/data/instance:
uuid: "9d0e72021aa443289c0bb45de789d382"
format_stamp: "Formatted at 2026-08-06 22:46:54 on dist-test-slave-zh2d"
server_key: "a4b1e9ee8bab5c3627561d3cf97755d2"
server_key_iv: "60ef950bcce8310c8b4e9e14cd5cdd73"
server_key_version: "encryptionkey@0"
I20260806 22:46:54.793398 8669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/minicluster-data/master-0/wal/instance:
uuid: "9d0e72021aa443289c0bb45de789d382"
format_stamp: "Formatted at 2026-08-06 22:46:54 on dist-test-slave-zh2d"
server_key: "a4b1e9ee8bab5c3627561d3cf97755d2"
server_key_iv: "60ef950bcce8310c8b4e9e14cd5cdd73"
server_key_version: "encryptionkey@0"
I20260806 22:46:54.794706 8669 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260806 22:46:54.795451 8695 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:54.795629 8669 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 22:46:54.795738 8669 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/minicluster-data/master-0/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/minicluster-data/master-0/wal
uuid: "9d0e72021aa443289c0bb45de789d382"
format_stamp: "Formatted at 2026-08-06 22:46:54 on dist-test-slave-zh2d"
server_key: "a4b1e9ee8bab5c3627561d3cf97755d2"
server_key_iv: "60ef950bcce8310c8b4e9e14cd5cdd73"
server_key_version: "encryptionkey@0"
I20260806 22:46:54.795815 8669 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:46:54.802803 8669 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:46:54.804621 8669 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:46:54.804795 8669 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:46:54.809895 8669 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.126:39721
I20260806 22:46:54.809939 8778 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.126:39721 every 8 connection(s)
I20260806 22:46:54.810266 8669 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
I20260806 22:46:54.811003 8779 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:46:54.813421 8779 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d0e72021aa443289c0bb45de789d382: Bootstrap starting.
I20260806 22:46:54.814038 8779 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9d0e72021aa443289c0bb45de789d382: Neither blocks nor log segments found. Creating new log.
I20260806 22:46:54.814353 8779 log.cc:824] T 00000000000000000000000000000000 P 9d0e72021aa443289c0bb45de789d382: Log is configured to *not* fsync() on all Append() calls
I20260806 22:46:54.815059 8779 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d0e72021aa443289c0bb45de789d382: No bootstrap required, opened a new log
I20260806 22:46:54.817163 8779 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9d0e72021aa443289c0bb45de789d382 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d0e72021aa443289c0bb45de789d382" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 39721 } }
I20260806 22:46:54.817298 8779 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9d0e72021aa443289c0bb45de789d382 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:46:54.817322 8779 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9d0e72021aa443289c0bb45de789d382 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d0e72021aa443289c0bb45de789d382, State: Initialized, Role: FOLLOWER
I20260806 22:46:54.817338 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 8669
I20260806 22:46:54.817476 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/minicluster-data/master-0/wal/instance
I20260806 22:46:54.817533 8779 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9d0e72021aa443289c0bb45de789d382 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d0e72021aa443289c0bb45de789d382" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 39721 } }
I20260806 22:46:54.817616 8779 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9d0e72021aa443289c0bb45de789d382 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:46:54.817659 8779 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9d0e72021aa443289c0bb45de789d382 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:46:54.817664 6305 external_mini_cluster.cc:1428] Setting key 8e9bc3c4a181761c0d7c3716d35d7ff8
I20260806 22:46:54.817709 8779 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9d0e72021aa443289c0bb45de789d382 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:46:54.818377 8779 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9d0e72021aa443289c0bb45de789d382 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d0e72021aa443289c0bb45de789d382" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 39721 } }
I20260806 22:46:54.818507 8779 leader_election.cc:302] T 00000000000000000000000000000000 P 9d0e72021aa443289c0bb45de789d382 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9d0e72021aa443289c0bb45de789d382; no voters:
I20260806 22:46:54.818684 8779 leader_election.cc:288] T 00000000000000000000000000000000 P 9d0e72021aa443289c0bb45de789d382 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:46:54.818723 8786 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9d0e72021aa443289c0bb45de789d382 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:46:54.818877 8786 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9d0e72021aa443289c0bb45de789d382 [term 1 LEADER]: Becoming Leader. State: Replica: 9d0e72021aa443289c0bb45de789d382, State: Running, Role: LEADER
I20260806 22:46:54.818975 8779 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9d0e72021aa443289c0bb45de789d382 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:46:54.818982 8786 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9d0e72021aa443289c0bb45de789d382 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d0e72021aa443289c0bb45de789d382" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 39721 } }
I20260806 22:46:54.819532 8786 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d0e72021aa443289c0bb45de789d382 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9d0e72021aa443289c0bb45de789d382. Latest consensus state: current_term: 1 leader_uuid: "9d0e72021aa443289c0bb45de789d382" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d0e72021aa443289c0bb45de789d382" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 39721 } } }
I20260806 22:46:54.819630 8786 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d0e72021aa443289c0bb45de789d382 [sys.catalog]: This master's current role is: LEADER
I20260806 22:46:54.819542 8788 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d0e72021aa443289c0bb45de789d382 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9d0e72021aa443289c0bb45de789d382" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d0e72021aa443289c0bb45de789d382" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 39721 } } }
I20260806 22:46:54.819864 8788 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d0e72021aa443289c0bb45de789d382 [sys.catalog]: This master's current role is: LEADER
I20260806 22:46:54.819855 8798 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:46:54.820542 8798 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:46:54.822348 8798 catalog_manager.cc:1347] Generated new cluster ID: 9f91e5785e5c4f70bd0ebac0ae9c038f
I20260806 22:46:54.822407 8798 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 22:46:54 dist-test-slave-zh2d krb5kdc[8654](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056414, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
I20260806 22:46:54.829404 8784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:54.818196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59632 (local address 127.6.40.126:39721)
0806 22:46:54.818387 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:54.818399 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:54.818435 (+ 36us) server_negotiation.cc:407] Connection header received
0806 22:46:54.818570 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:54.818576 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:54.818637 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:54.819628 (+ 991us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:54.819734 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:54.821224 (+ 1490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:54.822364 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:54.822845 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:54.822971 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:54.825444 (+ 2473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:54.825449 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:54.825449 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:46:54.825461 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:54.827770 (+ 2309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:54.828175 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:54.828178 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:54.828179 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:54.828286 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:54.828515 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:54.828516 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:54.828517 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:54.828767 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:54.828989 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:54.829060 (+ 71us) server_negotiation.cc:299] Negotiation successful
0806 22:46:54.829213 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":82,"threads_started":1}
I20260806 22:46:54.835933 8798 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:46:54.836457 8798 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:46:54.842433 8798 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9d0e72021aa443289c0bb45de789d382: Generated new TSK 0
I20260806 22:46:54.842635 8798 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.40.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.40.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.40.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.65.keytab.
Entry for principal kudu/127.6.40.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.65.keytab.
I20260806 22:46:54.879663 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.65:0
--local_ip_for_outbound_sockets=127.6.40.65
--webserver_interface=127.6.40.65
--webserver_port=0
--tserver_master_addrs=127.6.40.126:39721
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.65.keytab
--principal=kudu/127.6.40.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:36477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:46:54.972000 8816 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:46:54.972213 8816 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:46:54.972242 8816 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:46:54.973402 8816 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:46:54.973439 8816 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:46:54.973460 8816 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 22:46:54.973472 8816 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 22:46:54.973489 8816 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.65
I20260806 22:46:54.974923 8816 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:36477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.65.keytab
--principal=kudu/127.6.40.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.40.65
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.40.126:39721
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8816
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.65
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:46:54.975109 8816 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:46:54.975292 8816 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:46:54.980938 8826 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:54.980996 8827 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:46:54.981092 8816 server_base.cc:1034] running on GCE node
W20260806 22:46:54.980928 8829 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:46:54.981318 8816 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:46:54.981603 8816 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:46:54.982771 8816 hybrid_clock.cc:648] HybridClock initialized: now 1786056414982766 us; error 41 us; skew 500 ppm
Aug 06 22:46:54 dist-test-slave-zh2d krb5kdc[8654](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056414, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:46:54.985459 8816 init.cc:377] Logged in from keytab as kudu/127.6.40.65@KRBTEST.COM (short username kudu)
I20260806 22:46:54.985901 8816 webserver.cc:460] Webserver started at http://127.6.40.65:37241/ using document root <none> and password file <none>
I20260806 22:46:54.986131 8816 fs_manager.cc:359] Metadata directory not provided
I20260806 22:46:54.986198 8816 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:46:54.986352 8816 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:46:54.987421 8816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/minicluster-data/ts-0/data/instance:
uuid: "4d6b767d87784ec3a9f8ebb281ae463c"
format_stamp: "Formatted at 2026-08-06 22:46:54 on dist-test-slave-zh2d"
server_key: "0a7e0a6e07de31fa89ae5d81b0c75052"
server_key_iv: "e69af4b5f345af15fbe45054fdc0cc9f"
server_key_version: "encryptionkey@0"
I20260806 22:46:54.987845 8816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/minicluster-data/ts-0/wal/instance:
uuid: "4d6b767d87784ec3a9f8ebb281ae463c"
format_stamp: "Formatted at 2026-08-06 22:46:54 on dist-test-slave-zh2d"
server_key: "0a7e0a6e07de31fa89ae5d81b0c75052"
server_key_iv: "e69af4b5f345af15fbe45054fdc0cc9f"
server_key_version: "encryptionkey@0"
I20260806 22:46:54.989238 8816 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:46:54.990012 8837 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:54.990188 8816 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:46:54.990295 8816 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/minicluster-data/ts-0/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/minicluster-data/ts-0/wal
uuid: "4d6b767d87784ec3a9f8ebb281ae463c"
format_stamp: "Formatted at 2026-08-06 22:46:54 on dist-test-slave-zh2d"
server_key: "0a7e0a6e07de31fa89ae5d81b0c75052"
server_key_iv: "e69af4b5f345af15fbe45054fdc0cc9f"
server_key_version: "encryptionkey@0"
I20260806 22:46:54.990370 8816 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:46:55.006986 8816 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:46:55.009410 8816 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:46:55.009585 8816 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:46:55.010170 8816 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:46:55.010237 8816 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:55.010286 8816 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:46:55.010324 8816 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 22:46:55 dist-test-slave-zh2d krb5kdc[8654](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056414, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.65@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
I20260806 22:46:55.017956 8816 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.65:33289
I20260806 22:46:55.018081 8987 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.65:33289 every 8 connection(s)
I20260806 22:46:55.018374 8816 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
I20260806 22:46:55.021603 8784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:55.010738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.65:45923 (local address 127.6.40.126:39721)
0806 22:46:55.010834 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:55.010837 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:55.011017 (+ 180us) server_negotiation.cc:407] Connection header received
0806 22:46:55.011672 (+ 655us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:55.011675 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:55.011692 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:55.011788 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:55.012677 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:55.013572 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:55.015272 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:55.015658 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:55.015739 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:55.018494 (+ 2755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:55.018496 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:55.018497 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:46:55.018506 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:55.020279 (+ 1773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:55.020669 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:55.020670 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:55.020670 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:55.020731 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:55.021044 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:55.021045 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:55.021045 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:55.021157 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:55.021215 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:55.021462 (+ 247us) server_negotiation.cc:299] Negotiation successful
0806 22:46:55.021535 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 22:46:55.021781 8849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:55.010687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:39721 (local address 127.6.40.65:45923)
0806 22:46:55.010879 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:55.010920 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:55.011553 (+ 633us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:55.011882 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:55.011887 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:55.012230 (+ 343us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:46:55.012557 (+ 327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:55.012563 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:55.013702 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:55.013704 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:55.015153 (+ 1449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:55.015155 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:55.015773 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:55.015774 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:55.015924 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:55.016494 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:46:55.016523 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:46:55.018366 (+ 1843us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:46:55.020353 (+ 1987us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:55.020358 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:55.020360 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:55.020578 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:55.020806 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:55.020807 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:55.020807 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:55.020981 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:55.021219 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:46:55.021225 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:46:55.021344 (+ 119us) client_negotiation.cc:769] Sending connection context
0806 22:46:55.021427 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 22:46:55.021556 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":107,"threads_started":1}
I20260806 22:46:55.022609 8989 heartbeater.cc:347] Connected to a master server at 127.6.40.126:39721
I20260806 22:46:55.022701 8989 heartbeater.cc:464] Registering TS with master...
I20260806 22:46:55.022907 8989 heartbeater.cc:511] Master 127.6.40.126:39721 requested a full tablet report, sending...
I20260806 22:46:55.023505 8716 ts_manager.cc:194] Registered new tserver with Master: 4d6b767d87784ec3a9f8ebb281ae463c (127.6.40.65:33289)
I20260806 22:46:55.024171 8716 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.40.65@KRBTEST.COM'} at 127.6.40.65:45923
I20260806 22:46:55.025734 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 8816
I20260806 22:46:55.025852 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/minicluster-data/ts-0/wal/instance
I20260806 22:46:55.026026 6305 external_mini_cluster.cc:1428] Setting key 205420442df41bd0a38477ab9aed7a78
I20260806 22:46:55.029883 8997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:55.023673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:39721 (local address 127.6.40.65:34413)
0806 22:46:55.023823 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:55.023832 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:55.023911 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:55.024032 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:55.024033 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:55.024211 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:46:55.024294 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:55.024296 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:55.025199 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:55.025200 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:55.025733 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:55.025735 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:55.026210 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:55.026211 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:55.026292 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:55.026744 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:46:55.026748 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:46:55.027016 (+ 268us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:46:55.028950 (+ 1934us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:55.028951 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:55.028952 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:55.029119 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:55.029328 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:55.029329 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:55.029329 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:55.029379 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:55.029653 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:46:55.029654 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:46:55.029690 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 22:46:55.029740 (+ 50us) client_negotiation.cc:240] Negotiation successful
0806 22:46:55.029818 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":88,"threads_started":1}
I20260806 22:46:55.029886 8784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:55.023720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.65:34413 (local address 127.6.40.126:39721)
0806 22:46:55.023766 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:55.023768 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:55.023920 (+ 152us) server_negotiation.cc:407] Connection header received
0806 22:46:55.023965 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:55.023966 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:55.023976 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:55.024021 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:55.024415 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:55.025115 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:55.025831 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:55.026138 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:55.026198 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:55.027106 (+ 908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:55.027115 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:55.027115 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:46:55.027122 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:55.028825 (+ 1703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:55.029194 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:55.029195 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:55.029195 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:55.029260 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:55.029443 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:55.029444 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:55.029444 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:55.029588 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:55.029658 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:55.029765 (+ 107us) server_negotiation.cc:299] Negotiation successful
0806 22:46:55.029828 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
WARNING: no policy specified for kudu/127.6.40.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.40.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.40.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.66.keytab.
Entry for principal kudu/127.6.40.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.66.keytab.
I20260806 22:46:55.053241 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.66:0
--local_ip_for_outbound_sockets=127.6.40.66
--webserver_interface=127.6.40.66
--webserver_port=0
--tserver_master_addrs=127.6.40.126:39721
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.66.keytab
--principal=kudu/127.6.40.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:36477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:46:55.139816 9002 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:46:55.140012 9002 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:46:55.140038 9002 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:46:55.141148 9002 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:46:55.141182 9002 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:46:55.141202 9002 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 22:46:55.141214 9002 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 22:46:55.141232 9002 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.66
I20260806 22:46:55.142613 9002 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:36477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.66.keytab
--principal=kudu/127.6.40.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.40.66
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.40.126:39721
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9002
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.66
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:46:55.142791 9002 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:46:55.142967 9002 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:46:55.148209 9012 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:46:55.148440 9002 server_base.cc:1034] running on GCE node
W20260806 22:46:55.148219 9010 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:55.148218 9009 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:46:55.148736 9002 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:46:55.149044 9002 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:46:55.150205 9002 hybrid_clock.cc:648] HybridClock initialized: now 1786056415150154 us; error 68 us; skew 500 ppm
Aug 06 22:46:55 dist-test-slave-zh2d krb5kdc[8654](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056415, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:46:55.152945 9002 init.cc:377] Logged in from keytab as kudu/127.6.40.66@KRBTEST.COM (short username kudu)
I20260806 22:46:55.153457 9002 webserver.cc:460] Webserver started at http://127.6.40.66:37105/ using document root <none> and password file <none>
I20260806 22:46:55.153687 9002 fs_manager.cc:359] Metadata directory not provided
I20260806 22:46:55.153731 9002 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:46:55.153880 9002 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:46:55.154968 9002 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/minicluster-data/ts-1/data/instance:
uuid: "a3ae77f560314bb69116291d0eb3f635"
format_stamp: "Formatted at 2026-08-06 22:46:55 on dist-test-slave-zh2d"
server_key: "74fc641e607e77ca99b3ee9cfec0c03a"
server_key_iv: "dbf4b2a9b90dc88f986aac4dac3f2595"
server_key_version: "encryptionkey@0"
I20260806 22:46:55.155340 9002 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/minicluster-data/ts-1/wal/instance:
uuid: "a3ae77f560314bb69116291d0eb3f635"
format_stamp: "Formatted at 2026-08-06 22:46:55 on dist-test-slave-zh2d"
server_key: "74fc641e607e77ca99b3ee9cfec0c03a"
server_key_iv: "dbf4b2a9b90dc88f986aac4dac3f2595"
server_key_version: "encryptionkey@0"
I20260806 22:46:55.156913 9002 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:46:55.157675 9020 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:55.157855 9002 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:46:55.157959 9002 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/minicluster-data/ts-1/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/minicluster-data/ts-1/wal
uuid: "a3ae77f560314bb69116291d0eb3f635"
format_stamp: "Formatted at 2026-08-06 22:46:55 on dist-test-slave-zh2d"
server_key: "74fc641e607e77ca99b3ee9cfec0c03a"
server_key_iv: "dbf4b2a9b90dc88f986aac4dac3f2595"
server_key_version: "encryptionkey@0"
I20260806 22:46:55.158044 9002 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:46:55.175276 9002 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:46:55.177500 9002 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:46:55.177664 9002 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:46:55.178256 9002 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:46:55.178318 9002 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:55.178364 9002 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:46:55.178401 9002 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 22:46:55 dist-test-slave-zh2d krb5kdc[8654](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056415, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.66@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
I20260806 22:46:55.186952 9002 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.66:46183
I20260806 22:46:55.187023 9179 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.66:46183 every 8 connection(s)
I20260806 22:46:55.187400 9002 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
I20260806 22:46:55.188774 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 9002
I20260806 22:46:55.188895 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/minicluster-data/ts-1/wal/instance
I20260806 22:46:55.189095 6305 external_mini_cluster.cc:1428] Setting key 5ed64e344a545de0b399c4b6d4eaea10
I20260806 22:46:55.189370 8784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:55.178740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.66:58279 (local address 127.6.40.126:39721)
0806 22:46:55.178812 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:55.178814 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:55.179069 (+ 255us) server_negotiation.cc:407] Connection header received
0806 22:46:55.179620 (+ 551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:55.179622 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:55.179637 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:55.179692 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:55.180554 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:55.181172 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:55.182224 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:55.182465 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:55.182533 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:55.186113 (+ 3580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:55.186115 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:55.186115 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:46:55.186124 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:55.187907 (+ 1783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:55.188312 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:55.188313 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:55.188314 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:55.188370 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:55.188630 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:55.188632 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:55.188632 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:55.188784 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:55.188887 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:55.189200 (+ 313us) server_negotiation.cc:299] Negotiation successful
0806 22:46:55.189291 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 22:46:55.189620 9040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:55.178723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:39721 (local address 127.6.40.66:58279)
0806 22:46:55.178904 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:55.178946 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:55.179505 (+ 559us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:55.179776 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:55.179780 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:55.180149 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:46:55.180460 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:55.180466 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:55.181268 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:55.181269 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:55.182126 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:55.182128 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:55.182570 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:55.182571 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:55.182754 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:55.183396 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:46:55.183407 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:46:55.185998 (+ 2591us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:46:55.188003 (+ 2005us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:55.188009 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:55.188010 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:55.188240 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:55.188435 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:55.188435 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:55.188436 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:55.188536 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:55.188908 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:46:55.188915 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:46:55.189052 (+ 137us) client_negotiation.cc:769] Sending connection context
0806 22:46:55.189167 (+ 115us) client_negotiation.cc:240] Negotiation successful
0806 22:46:55.189363 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":88,"threads_started":1}
I20260806 22:46:55.190126 9181 heartbeater.cc:347] Connected to a master server at 127.6.40.126:39721
I20260806 22:46:55.190217 9181 heartbeater.cc:464] Registering TS with master...
I20260806 22:46:55.190402 9181 heartbeater.cc:511] Master 127.6.40.126:39721 requested a full tablet report, sending...
I20260806 22:46:55.191256 8716 ts_manager.cc:194] Registered new tserver with Master: a3ae77f560314bb69116291d0eb3f635 (127.6.40.66:46183)
I20260806 22:46:55.191797 8716 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.40.66@KRBTEST.COM'} at 127.6.40.66:58279
I20260806 22:46:55.197808 8784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:55.191954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.66:44591 (local address 127.6.40.126:39721)
0806 22:46:55.191990 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:55.191992 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:55.192000 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:46:55.192019 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:55.192019 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:55.192028 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:55.192094 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:55.192554 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:55.193212 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:55.193906 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:55.194094 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:55.194153 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:55.195208 (+ 1055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:55.195209 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:55.195209 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:46:55.195215 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:55.196568 (+ 1353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:55.197025 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:55.197026 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:55.197026 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:55.197076 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:55.197288 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:55.197289 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:55.197289 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:55.197450 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:55.197503 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:55.197687 (+ 184us) server_negotiation.cc:299] Negotiation successful
0806 22:46:55.197746 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 22:46:55.197968 9190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:55.191411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:39721 (local address 127.6.40.66:44591)
0806 22:46:55.191583 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:55.191596 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:55.191674 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:55.192160 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:55.192161 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:55.192373 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:46:55.192455 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:55.192457 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:55.193320 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:55.193320 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:55.193839 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:55.193840 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:55.194172 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:55.194173 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:55.194285 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:55.194673 (+ 388us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:46:55.194678 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:46:55.195103 (+ 425us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:46:55.196660 (+ 1557us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:55.196662 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:55.196662 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:55.196943 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:55.197143 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:55.197144 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:55.197144 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:55.197217 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:55.197521 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:46:55.197522 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:46:55.197580 (+ 58us) client_negotiation.cc:769] Sending connection context
0806 22:46:55.197672 (+ 92us) client_negotiation.cc:240] Negotiation successful
0806 22:46:55.197800 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":114,"threads_started":1}
WARNING: no policy specified for kudu/127.6.40.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.40.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.40.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.67.keytab.
Entry for principal kudu/127.6.40.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.67.keytab.
I20260806 22:46:55.224629 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.67:0
--local_ip_for_outbound_sockets=127.6.40.67
--webserver_interface=127.6.40.67
--webserver_port=0
--tserver_master_addrs=127.6.40.126:39721
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.67.keytab
--principal=kudu/127.6.40.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:36477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:46:55.309155 9194 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:46:55.309353 9194 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:46:55.309379 9194 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:46:55.310469 9194 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:46:55.310503 9194 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:46:55.310523 9194 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 22:46:55.310535 9194 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 22:46:55.310551 9194 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.67
I20260806 22:46:55.312054 9194 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:36477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.67.keytab
--principal=kudu/127.6.40.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.40.67
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.40.126:39721
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9194
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.67
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:46:55.312249 9194 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:46:55.312438 9194 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:46:55.317795 9200 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:55.317806 9199 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:55.317770 9202 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:46:55.317996 9194 server_base.cc:1034] running on GCE node
I20260806 22:46:55.318181 9194 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:46:55.318436 9194 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:46:55.319577 9194 hybrid_clock.cc:648] HybridClock initialized: now 1786056415319561 us; error 36 us; skew 500 ppm
Aug 06 22:46:55 dist-test-slave-zh2d krb5kdc[8654](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056415, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:46:55.322376 9194 init.cc:377] Logged in from keytab as kudu/127.6.40.67@KRBTEST.COM (short username kudu)
I20260806 22:46:55.322906 9194 webserver.cc:460] Webserver started at http://127.6.40.67:37893/ using document root <none> and password file <none>
I20260806 22:46:55.323112 9194 fs_manager.cc:359] Metadata directory not provided
I20260806 22:46:55.323155 9194 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:46:55.323243 9194 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:46:55.324368 9194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/minicluster-data/ts-2/data/instance:
uuid: "982377b0de554faf89f0dbff3b55a8d0"
format_stamp: "Formatted at 2026-08-06 22:46:55 on dist-test-slave-zh2d"
server_key: "d7a60d420b59e31ef3d7b051752fc46d"
server_key_iv: "b0ddc11e622e10a783a4e32b60fb5807"
server_key_version: "encryptionkey@0"
I20260806 22:46:55.324671 9194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/minicluster-data/ts-2/wal/instance:
uuid: "982377b0de554faf89f0dbff3b55a8d0"
format_stamp: "Formatted at 2026-08-06 22:46:55 on dist-test-slave-zh2d"
server_key: "d7a60d420b59e31ef3d7b051752fc46d"
server_key_iv: "b0ddc11e622e10a783a4e32b60fb5807"
server_key_version: "encryptionkey@0"
I20260806 22:46:55.326071 9194 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:46:55.326889 9215 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:55.327104 9194 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:46:55.327198 9194 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/minicluster-data/ts-2/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/minicluster-data/ts-2/wal
uuid: "982377b0de554faf89f0dbff3b55a8d0"
format_stamp: "Formatted at 2026-08-06 22:46:55 on dist-test-slave-zh2d"
server_key: "d7a60d420b59e31ef3d7b051752fc46d"
server_key_iv: "b0ddc11e622e10a783a4e32b60fb5807"
server_key_version: "encryptionkey@0"
I20260806 22:46:55.327276 9194 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:46:55.347860 9194 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:46:55.350029 9194 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:46:55.350188 9194 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:46:55.350831 9194 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:46:55.350873 9194 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:55.350898 9194 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:46:55.350910 9194 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 22:46:55 dist-test-slave-zh2d krb5kdc[8654](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056415, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.67@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
I20260806 22:46:55.358902 9194 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.67:38615
I20260806 22:46:55.359051 9386 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.67:38615 every 8 connection(s)
I20260806 22:46:55.359380 9194 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
I20260806 22:46:55.360395 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 9194
I20260806 22:46:55.360502 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/minicluster-data/ts-2/wal/instance
I20260806 22:46:55.360739 6305 external_mini_cluster.cc:1428] Setting key fd8c27682173c934d9fd9a7b5f05ee47
I20260806 22:46:55.362188 8784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:55.351342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.67:60607 (local address 127.6.40.126:39721)
0806 22:46:55.351435 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:55.351438 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:55.351702 (+ 264us) server_negotiation.cc:407] Connection header received
0806 22:46:55.352271 (+ 569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:55.352272 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:55.352288 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:55.352370 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:55.353283 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:55.353904 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:55.355061 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:55.355286 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:55.355371 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:55.358359 (+ 2988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:55.358361 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:55.358361 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:46:55.358369 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:55.360348 (+ 1979us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:55.360945 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:55.360946 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:55.360947 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:55.361017 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:55.361334 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:55.361336 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:55.361336 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:55.361519 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:55.361601 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:55.361991 (+ 390us) server_negotiation.cc:299] Negotiation successful
0806 22:46:55.362102 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 22:46:55.362335 9228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:55.351306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:39721 (local address 127.6.40.67:60607)
0806 22:46:55.351539 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:55.351587 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:55.352157 (+ 570us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:55.352446 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:55.352450 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:55.352926 (+ 476us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:46:55.353173 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:55.353178 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:55.354007 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:55.354008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:55.354942 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:55.354945 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:55.355381 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:55.355382 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:55.355538 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:55.356148 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:46:55.356163 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:46:55.358254 (+ 2091us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:46:55.360511 (+ 2257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:55.360518 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:55.360520 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:55.360810 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:55.361111 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:55.361113 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:55.361113 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:55.361244 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:55.361613 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:46:55.361622 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:46:55.361803 (+ 181us) client_negotiation.cc:769] Sending connection context
0806 22:46:55.361913 (+ 110us) client_negotiation.cc:240] Negotiation successful
0806 22:46:55.362114 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":150,"threads_started":1}
I20260806 22:46:55.362877 9387 heartbeater.cc:347] Connected to a master server at 127.6.40.126:39721
I20260806 22:46:55.362970 9387 heartbeater.cc:464] Registering TS with master...
I20260806 22:46:55.363185 9387 heartbeater.cc:511] Master 127.6.40.126:39721 requested a full tablet report, sending...
I20260806 22:46:55.363566 8715 ts_manager.cc:194] Registered new tserver with Master: 982377b0de554faf89f0dbff3b55a8d0 (127.6.40.67:38615)
I20260806 22:46:55.364126 8715 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.40.67@KRBTEST.COM'} at 127.6.40.67:60607
I20260806 22:46:55.369753 8784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:55.364048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.67:60161 (local address 127.6.40.126:39721)
0806 22:46:55.364087 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:55.364088 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:55.364257 (+ 169us) server_negotiation.cc:407] Connection header received
0806 22:46:55.364291 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:55.364292 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:55.364303 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:55.364369 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:55.364745 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:55.365352 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:55.366075 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:55.366274 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:55.366329 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:55.367217 (+ 888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:55.367218 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:55.367219 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:46:55.367226 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:55.368784 (+ 1558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:55.369150 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:55.369151 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:55.369152 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:55.369206 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:55.369360 (+ 154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:55.369361 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:55.369361 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:55.369455 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:55.369505 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:55.369626 (+ 121us) server_negotiation.cc:299] Negotiation successful
0806 22:46:55.369687 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 22:46:55.369762 9395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:55.363974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:39721 (local address 127.6.40.67:60161)
0806 22:46:55.364154 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:55.364163 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:55.364247 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:55.364388 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:55.364389 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:55.364595 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:46:55.364660 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:55.364662 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:55.365473 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:55.365474 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:55.366000 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:55.366002 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:55.366341 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:55.366342 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:55.366404 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:55.366830 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:46:55.366835 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:46:55.367135 (+ 300us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:46:55.368907 (+ 1772us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:55.368908 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:55.368908 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:55.369082 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:55.369268 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:55.369269 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:55.369269 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:55.369320 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:55.369508 (+ 188us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:46:55.369508 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:46:55.369558 (+ 50us) client_negotiation.cc:769] Sending connection context
0806 22:46:55.369609 (+ 51us) client_negotiation.cc:240] Negotiation successful
0806 22:46:55.369692 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":86,"threads_started":1}
I20260806 22:46:55.373035 6305 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 22:46:55 dist-test-slave-zh2d krb5kdc[8654](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056415, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 22:46:55 dist-test-slave-zh2d krb5kdc[8654](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056415, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
I20260806 22:46:55.398525 8784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:55.390388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40564 (local address 127.6.40.126:39721)
0806 22:46:55.390470 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:55.390472 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:55.390505 (+ 33us) server_negotiation.cc:407] Connection header received
0806 22:46:55.390659 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:55.390661 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:55.390674 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:55.390742 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:55.391498 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:55.392052 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:55.392952 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:55.393163 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:55.393232 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:55.395616 (+ 2384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:55.395618 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:55.395618 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:46:55.395625 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:55.397238 (+ 1613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:55.397564 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:55.397565 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:55.397566 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:55.397622 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:55.397862 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:55.397880 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:55.397881 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:55.398017 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:55.398104 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:55.398213 (+ 109us) server_negotiation.cc:299] Negotiation successful
0806 22:46:55.398347 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 22:46:55.400290 8715 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40564:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:46:55.402424 8715 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:46:55.408985 9409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:55.405069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.67:38615 (local address 127.0.0.1:57548)
0806 22:46:55.405255 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:55.405298 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:55.405365 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:55.405846 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:55.405850 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:55.405853 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:46:55.405935 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:55.405938 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:55.406981 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:55.406982 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:55.407874 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:55.407877 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:55.408594 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:55.408595 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:55.408697 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:55.408712 (+ 15us) client_negotiation.cc:769] Sending connection context
0806 22:46:55.408796 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 22:46:55.408895 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":47,"threads_started":1}
I20260806 22:46:55.409272 9411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:55.405443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57548 (local address 127.6.40.67:38615)
0806 22:46:55.405657 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:55.405662 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:55.405673 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:46:55.405719 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:55.405721 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:55.405739 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:55.405836 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:46:55.406021 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:55.406896 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:55.407984 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:55.408505 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:55.408566 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:55.408607 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:55.408819 (+ 212us) server_negotiation.cc:299] Negotiation successful
0806 22:46:55.408912 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":103,"threads_started":1}
I20260806 22:46:55.410054 9293 tablet_service.cc:1467] Processing CreateTablet for tablet fd91884bdffe4d3685831649a184aa1a (DEFAULT_TABLE table=test-table [id=1db510231e024950a296d324388d16f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:46:55.410450 9293 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd91884bdffe4d3685831649a184aa1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:46:55.411247 9410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:55.405101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.65:33289 (local address 127.0.0.1:33792)
0806 22:46:55.405273 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:55.405311 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:55.405390 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:55.406277 (+ 887us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:55.406278 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:55.406281 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:46:55.406343 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:55.406345 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:55.407848 (+ 1503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:55.407849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:55.408813 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:55.408816 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:55.410962 (+ 2146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:55.410963 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:55.411034 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:55.411038 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:46:55.411101 (+ 63us) client_negotiation.cc:240] Negotiation successful
0806 22:46:55.411146 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"mutex_wait_us":3,"thread_start_us":104,"threads_started":1}
I20260806 22:46:55.411273 9412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:55.405518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33792 (local address 127.6.40.65:33289)
0806 22:46:55.405736 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:55.405742 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:55.405755 (+ 13us) server_negotiation.cc:407] Connection header received
0806 22:46:55.406094 (+ 339us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:55.406097 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:55.406116 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:55.406203 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:46:55.406448 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:55.407725 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:55.410111 (+ 2386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:55.410862 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:55.410945 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:55.411009 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:55.411125 (+ 116us) server_negotiation.cc:299] Negotiation successful
0806 22:46:55.411171 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":116,"threads_started":1}
I20260806 22:46:55.411350 9408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:55.405131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.66:46183 (local address 127.0.0.1:51600)
0806 22:46:55.406056 (+ 925us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:55.406068 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:55.406137 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:55.406323 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:55.406324 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:55.406326 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:46:55.406372 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:55.406374 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:55.407845 (+ 1471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:55.407846 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:55.409992 (+ 2146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:55.409996 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:55.411118 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:55.411119 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:55.411173 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:55.411176 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:46:55.411227 (+ 51us) client_negotiation.cc:240] Negotiation successful
0806 22:46:55.411264 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":912,"mutex_wait_us":1038,"thread_start_us":101,"threads_started":1}
I20260806 22:46:55.411633 9414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:55.405768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51600 (local address 127.6.40.66:46183)
0806 22:46:55.406006 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:55.406011 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:55.406139 (+ 128us) server_negotiation.cc:407] Connection header received
0806 22:46:55.406209 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:55.406211 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:55.406228 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:55.406320 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:46:55.406439 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:55.407731 (+ 1292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:55.410108 (+ 2377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:55.410931 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:55.411011 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:55.411080 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:55.411431 (+ 351us) server_negotiation.cc:299] Negotiation successful
0806 22:46:55.411484 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":85,"threads_started":1}
I20260806 22:46:55.413600 9098 tablet_service.cc:1467] Processing CreateTablet for tablet fd91884bdffe4d3685831649a184aa1a (DEFAULT_TABLE table=test-table [id=1db510231e024950a296d324388d16f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:46:55.413792 9415 tablet_bootstrap.cc:492] T fd91884bdffe4d3685831649a184aa1a P 982377b0de554faf89f0dbff3b55a8d0: Bootstrap starting.
I20260806 22:46:55.413810 8902 tablet_service.cc:1467] Processing CreateTablet for tablet fd91884bdffe4d3685831649a184aa1a (DEFAULT_TABLE table=test-table [id=1db510231e024950a296d324388d16f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:46:55.413980 9098 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd91884bdffe4d3685831649a184aa1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:46:55.414160 8902 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd91884bdffe4d3685831649a184aa1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:46:55.414690 9415 tablet_bootstrap.cc:654] T fd91884bdffe4d3685831649a184aa1a P 982377b0de554faf89f0dbff3b55a8d0: Neither blocks nor log segments found. Creating new log.
I20260806 22:46:55.415079 9415 log.cc:824] T fd91884bdffe4d3685831649a184aa1a P 982377b0de554faf89f0dbff3b55a8d0: Log is configured to *not* fsync() on all Append() calls
I20260806 22:46:55.415979 9415 tablet_bootstrap.cc:492] T fd91884bdffe4d3685831649a184aa1a P 982377b0de554faf89f0dbff3b55a8d0: No bootstrap required, opened a new log
I20260806 22:46:55.416039 9415 ts_tablet_manager.cc:1397] T fd91884bdffe4d3685831649a184aa1a P 982377b0de554faf89f0dbff3b55a8d0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:46:55.417598 9418 tablet_bootstrap.cc:492] T fd91884bdffe4d3685831649a184aa1a P 4d6b767d87784ec3a9f8ebb281ae463c: Bootstrap starting.
I20260806 22:46:55.417917 9419 tablet_bootstrap.cc:492] T fd91884bdffe4d3685831649a184aa1a P a3ae77f560314bb69116291d0eb3f635: Bootstrap starting.
I20260806 22:46:55.418236 9415 raft_consensus.cc:348] T fd91884bdffe4d3685831649a184aa1a P 982377b0de554faf89f0dbff3b55a8d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "982377b0de554faf89f0dbff3b55a8d0" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 38615 } } peers { permanent_uuid: "4d6b767d87784ec3a9f8ebb281ae463c" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 33289 } } peers { permanent_uuid: "a3ae77f560314bb69116291d0eb3f635" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 46183 } }
I20260806 22:46:55.418391 9415 raft_consensus.cc:374] T fd91884bdffe4d3685831649a184aa1a P 982377b0de554faf89f0dbff3b55a8d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:46:55.418437 9415 raft_consensus.cc:729] T fd91884bdffe4d3685831649a184aa1a P 982377b0de554faf89f0dbff3b55a8d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 982377b0de554faf89f0dbff3b55a8d0, State: Initialized, Role: FOLLOWER
I20260806 22:46:55.418571 9418 tablet_bootstrap.cc:654] T fd91884bdffe4d3685831649a184aa1a P 4d6b767d87784ec3a9f8ebb281ae463c: Neither blocks nor log segments found. Creating new log.
I20260806 22:46:55.418581 9415 consensus_queue.cc:260] T fd91884bdffe4d3685831649a184aa1a P 982377b0de554faf89f0dbff3b55a8d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "982377b0de554faf89f0dbff3b55a8d0" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 38615 } } peers { permanent_uuid: "4d6b767d87784ec3a9f8ebb281ae463c" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 33289 } } peers { permanent_uuid: "a3ae77f560314bb69116291d0eb3f635" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 46183 } }
I20260806 22:46:55.418643 9419 tablet_bootstrap.cc:654] T fd91884bdffe4d3685831649a184aa1a P a3ae77f560314bb69116291d0eb3f635: Neither blocks nor log segments found. Creating new log.
I20260806 22:46:55.418854 9415 ts_tablet_manager.cc:1428] T fd91884bdffe4d3685831649a184aa1a P 982377b0de554faf89f0dbff3b55a8d0: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:46:55.419013 9387 heartbeater.cc:503] Master 127.6.40.126:39721 was elected leader, sending a full tablet report...
I20260806 22:46:55.419054 9418 log.cc:824] T fd91884bdffe4d3685831649a184aa1a P 4d6b767d87784ec3a9f8ebb281ae463c: Log is configured to *not* fsync() on all Append() calls
I20260806 22:46:55.419085 9419 log.cc:824] T fd91884bdffe4d3685831649a184aa1a P a3ae77f560314bb69116291d0eb3f635: Log is configured to *not* fsync() on all Append() calls
I20260806 22:46:55.420037 9418 tablet_bootstrap.cc:492] T fd91884bdffe4d3685831649a184aa1a P 4d6b767d87784ec3a9f8ebb281ae463c: No bootstrap required, opened a new log
I20260806 22:46:55.420146 9418 ts_tablet_manager.cc:1397] T fd91884bdffe4d3685831649a184aa1a P 4d6b767d87784ec3a9f8ebb281ae463c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 22:46:55.420168 9419 tablet_bootstrap.cc:492] T fd91884bdffe4d3685831649a184aa1a P a3ae77f560314bb69116291d0eb3f635: No bootstrap required, opened a new log
I20260806 22:46:55.420236 9419 ts_tablet_manager.cc:1397] T fd91884bdffe4d3685831649a184aa1a P a3ae77f560314bb69116291d0eb3f635: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:46:55.422025 9419 raft_consensus.cc:348] T fd91884bdffe4d3685831649a184aa1a P a3ae77f560314bb69116291d0eb3f635 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "982377b0de554faf89f0dbff3b55a8d0" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 38615 } } peers { permanent_uuid: "4d6b767d87784ec3a9f8ebb281ae463c" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 33289 } } peers { permanent_uuid: "a3ae77f560314bb69116291d0eb3f635" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 46183 } }
I20260806 22:46:55.422173 9419 raft_consensus.cc:374] T fd91884bdffe4d3685831649a184aa1a P a3ae77f560314bb69116291d0eb3f635 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:46:55.422219 9419 raft_consensus.cc:729] T fd91884bdffe4d3685831649a184aa1a P a3ae77f560314bb69116291d0eb3f635 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3ae77f560314bb69116291d0eb3f635, State: Initialized, Role: FOLLOWER
I20260806 22:46:55.422333 9419 consensus_queue.cc:260] T fd91884bdffe4d3685831649a184aa1a P a3ae77f560314bb69116291d0eb3f635 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "982377b0de554faf89f0dbff3b55a8d0" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 38615 } } peers { permanent_uuid: "4d6b767d87784ec3a9f8ebb281ae463c" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 33289 } } peers { permanent_uuid: "a3ae77f560314bb69116291d0eb3f635" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 46183 } }
I20260806 22:46:55.422399 9418 raft_consensus.cc:348] T fd91884bdffe4d3685831649a184aa1a P 4d6b767d87784ec3a9f8ebb281ae463c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "982377b0de554faf89f0dbff3b55a8d0" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 38615 } } peers { permanent_uuid: "4d6b767d87784ec3a9f8ebb281ae463c" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 33289 } } peers { permanent_uuid: "a3ae77f560314bb69116291d0eb3f635" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 46183 } }
I20260806 22:46:55.422592 9418 raft_consensus.cc:374] T fd91884bdffe4d3685831649a184aa1a P 4d6b767d87784ec3a9f8ebb281ae463c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:46:55.422642 9181 heartbeater.cc:503] Master 127.6.40.126:39721 was elected leader, sending a full tablet report...
I20260806 22:46:55.422642 9418 raft_consensus.cc:729] T fd91884bdffe4d3685831649a184aa1a P 4d6b767d87784ec3a9f8ebb281ae463c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d6b767d87784ec3a9f8ebb281ae463c, State: Initialized, Role: FOLLOWER
I20260806 22:46:55.422765 9418 consensus_queue.cc:260] T fd91884bdffe4d3685831649a184aa1a P 4d6b767d87784ec3a9f8ebb281ae463c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "982377b0de554faf89f0dbff3b55a8d0" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 38615 } } peers { permanent_uuid: "4d6b767d87784ec3a9f8ebb281ae463c" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 33289 } } peers { permanent_uuid: "a3ae77f560314bb69116291d0eb3f635" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 46183 } }
I20260806 22:46:55.422890 9419 ts_tablet_manager.cc:1428] T fd91884bdffe4d3685831649a184aa1a P a3ae77f560314bb69116291d0eb3f635: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260806 22:46:55.423230 9418 ts_tablet_manager.cc:1428] T fd91884bdffe4d3685831649a184aa1a P 4d6b767d87784ec3a9f8ebb281ae463c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:46:55.423240 8989 heartbeater.cc:503] Master 127.6.40.126:39721 was elected leader, sending a full tablet report...
W20260806 22:46:55.438045 9185 tablet.cc:2369] T fd91884bdffe4d3685831649a184aa1a P a3ae77f560314bb69116291d0eb3f635: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:46:55.512252 9420 raft_consensus.cc:482] T fd91884bdffe4d3685831649a184aa1a P 982377b0de554faf89f0dbff3b55a8d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:46:55.512462 9420 raft_consensus.cc:504] T fd91884bdffe4d3685831649a184aa1a P 982377b0de554faf89f0dbff3b55a8d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "982377b0de554faf89f0dbff3b55a8d0" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 38615 } } peers { permanent_uuid: "4d6b767d87784ec3a9f8ebb281ae463c" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 33289 } } peers { permanent_uuid: "a3ae77f560314bb69116291d0eb3f635" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 46183 } }
I20260806 22:46:55.512945 9420 leader_election.cc:288] T fd91884bdffe4d3685831649a184aa1a P 982377b0de554faf89f0dbff3b55a8d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4d6b767d87784ec3a9f8ebb281ae463c (127.6.40.65:33289), a3ae77f560314bb69116291d0eb3f635 (127.6.40.66:46183)
I20260806 22:46:55.516583 9412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:55.513394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.67:42017 (local address 127.6.40.65:33289)
0806 22:46:55.513444 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:55.513446 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:55.513457 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:46:55.513527 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:55.513558 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:55.513573 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:55.513677 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:46:55.513986 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:55.514614 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:55.515736 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:55.516231 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:55.516287 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:55.516297 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:55.516492 (+ 195us) server_negotiation.cc:299] Negotiation successful
0806 22:46:55.516525 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 22:46:55.516630 9228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:55.513260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.65:33289 (local address 127.6.40.67:42017)
0806 22:46:55.513357 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:55.513372 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:55.513445 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:55.513737 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:55.513739 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:55.513742 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:46:55.513853 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:55.513856 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:55.514706 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:55.514707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:55.515639 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:55.515641 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:55.516310 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:55.516311 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:55.516407 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:55.516412 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:46:55.516478 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 22:46:55.516523 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71}
I20260806 22:46:55.517102 8933 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd91884bdffe4d3685831649a184aa1a" candidate_uuid: "982377b0de554faf89f0dbff3b55a8d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d6b767d87784ec3a9f8ebb281ae463c" is_pre_election: true
I20260806 22:46:55.517287 8933 raft_consensus.cc:2454] T fd91884bdffe4d3685831649a184aa1a P 4d6b767d87784ec3a9f8ebb281ae463c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 982377b0de554faf89f0dbff3b55a8d0 in term 0.
I20260806 22:46:55.517411 9414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:55.513378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.67:49201 (local address 127.6.40.66:46183)
0806 22:46:55.513435 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:55.513437 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:55.513720 (+ 283us) server_negotiation.cc:407] Connection header received
0806 22:46:55.513764 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:55.513766 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:55.513786 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:55.513868 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:46:55.514205 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:55.514904 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:55.516031 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:55.516557 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:55.516626 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:55.516637 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:55.517263 (+ 626us) server_negotiation.cc:299] Negotiation successful
0806 22:46:55.517322 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 22:46:55.517443 9425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:55.513303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.66:46183 (local address 127.6.40.67:49201)
0806 22:46:55.513618 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:55.513630 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:55.513706 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:55.513870 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:55.513871 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:55.513872 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:46:55.514093 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:55.514096 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:55.514993 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:55.514994 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:55.515938 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:55.515941 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:55.516869 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:55.516872 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:55.517146 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:55.517151 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:46:55.517233 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 22:46:55.517282 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":234,"threads_started":1}
I20260806 22:46:55.517553 9221 leader_election.cc:302] T fd91884bdffe4d3685831649a184aa1a P 982377b0de554faf89f0dbff3b55a8d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d6b767d87784ec3a9f8ebb281ae463c, 982377b0de554faf89f0dbff3b55a8d0; no voters:
I20260806 22:46:55.517679 9420 raft_consensus.cc:2781] T fd91884bdffe4d3685831649a184aa1a P 982377b0de554faf89f0dbff3b55a8d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:46:55.517738 9420 raft_consensus.cc:482] T fd91884bdffe4d3685831649a184aa1a P 982377b0de554faf89f0dbff3b55a8d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:46:55.517776 9420 raft_consensus.cc:3037] T fd91884bdffe4d3685831649a184aa1a P 982377b0de554faf89f0dbff3b55a8d0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:46:55.517853 9125 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd91884bdffe4d3685831649a184aa1a" candidate_uuid: "982377b0de554faf89f0dbff3b55a8d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3ae77f560314bb69116291d0eb3f635" is_pre_election: true
I20260806 22:46:55.518046 9125 raft_consensus.cc:2454] T fd91884bdffe4d3685831649a184aa1a P a3ae77f560314bb69116291d0eb3f635 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 982377b0de554faf89f0dbff3b55a8d0 in term 0.
I20260806 22:46:55.518916 9420 raft_consensus.cc:504] T fd91884bdffe4d3685831649a184aa1a P 982377b0de554faf89f0dbff3b55a8d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "982377b0de554faf89f0dbff3b55a8d0" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 38615 } } peers { permanent_uuid: "4d6b767d87784ec3a9f8ebb281ae463c" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 33289 } } peers { permanent_uuid: "a3ae77f560314bb69116291d0eb3f635" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 46183 } }
I20260806 22:46:55.519083 9420 leader_election.cc:288] T fd91884bdffe4d3685831649a184aa1a P 982377b0de554faf89f0dbff3b55a8d0 [CANDIDATE]: Term 1 election: Requested vote from peers 4d6b767d87784ec3a9f8ebb281ae463c (127.6.40.65:33289), a3ae77f560314bb69116291d0eb3f635 (127.6.40.66:46183)
W20260806 22:46:55.519222 8990 tablet.cc:2369] T fd91884bdffe4d3685831649a184aa1a P 4d6b767d87784ec3a9f8ebb281ae463c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:46:55.519238 8933 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd91884bdffe4d3685831649a184aa1a" candidate_uuid: "982377b0de554faf89f0dbff3b55a8d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d6b767d87784ec3a9f8ebb281ae463c"
I20260806 22:46:55.519292 9125 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd91884bdffe4d3685831649a184aa1a" candidate_uuid: "982377b0de554faf89f0dbff3b55a8d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3ae77f560314bb69116291d0eb3f635"
I20260806 22:46:55.519354 8933 raft_consensus.cc:3037] T fd91884bdffe4d3685831649a184aa1a P 4d6b767d87784ec3a9f8ebb281ae463c [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:46:55.519367 9125 raft_consensus.cc:3037] T fd91884bdffe4d3685831649a184aa1a P a3ae77f560314bb69116291d0eb3f635 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:46:55.520514 8933 raft_consensus.cc:2454] T fd91884bdffe4d3685831649a184aa1a P 4d6b767d87784ec3a9f8ebb281ae463c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 982377b0de554faf89f0dbff3b55a8d0 in term 1.
I20260806 22:46:55.520699 9221 leader_election.cc:302] T fd91884bdffe4d3685831649a184aa1a P 982377b0de554faf89f0dbff3b55a8d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d6b767d87784ec3a9f8ebb281ae463c, 982377b0de554faf89f0dbff3b55a8d0; no voters:
I20260806 22:46:55.520782 9420 raft_consensus.cc:2781] T fd91884bdffe4d3685831649a184aa1a P 982377b0de554faf89f0dbff3b55a8d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:46:55.520946 9420 raft_consensus.cc:686] T fd91884bdffe4d3685831649a184aa1a P 982377b0de554faf89f0dbff3b55a8d0 [term 1 LEADER]: Becoming Leader. State: Replica: 982377b0de554faf89f0dbff3b55a8d0, State: Running, Role: LEADER
I20260806 22:46:55.521051 9125 raft_consensus.cc:2454] T fd91884bdffe4d3685831649a184aa1a P a3ae77f560314bb69116291d0eb3f635 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 982377b0de554faf89f0dbff3b55a8d0 in term 1.
I20260806 22:46:55.521056 9420 consensus_queue.cc:237] T fd91884bdffe4d3685831649a184aa1a P 982377b0de554faf89f0dbff3b55a8d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "982377b0de554faf89f0dbff3b55a8d0" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 38615 } } peers { permanent_uuid: "4d6b767d87784ec3a9f8ebb281ae463c" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 33289 } } peers { permanent_uuid: "a3ae77f560314bb69116291d0eb3f635" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 46183 } }
I20260806 22:46:55.522022 8715 catalog_manager.cc:5495] T fd91884bdffe4d3685831649a184aa1a P 982377b0de554faf89f0dbff3b55a8d0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 982377b0de554faf89f0dbff3b55a8d0 (127.6.40.67). New cstate: current_term: 1 leader_uuid: "982377b0de554faf89f0dbff3b55a8d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "982377b0de554faf89f0dbff3b55a8d0" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 38615 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4d6b767d87784ec3a9f8ebb281ae463c" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 33289 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3ae77f560314bb69116291d0eb3f635" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 46183 } health_report { overall_health: UNKNOWN } } }
Aug 06 22:46:55 dist-test-slave-zh2d krb5kdc[8654](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056414, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.126@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
I20260806 22:46:55.573246 9435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:55.565789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:39721 (local address 127.0.0.1:40576)
0806 22:46:55.566024 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:55.566035 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:55.566106 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:55.566235 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:55.566236 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:55.566573 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:46:55.566627 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:55.566630 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:55.567276 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:55.567277 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:55.567934 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:55.567936 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:55.568275 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:55.568276 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:55.568323 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:55.569037 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:46:55.569042 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:46:55.570671 (+ 1629us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:46:55.572158 (+ 1487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:55.572166 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:55.572168 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:55.572382 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:55.572596 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:55.572596 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:55.572597 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:55.572656 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:55.572919 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:46:55.572926 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:46:55.572972 (+ 46us) client_negotiation.cc:769] Sending connection context
0806 22:46:55.573034 (+ 62us) client_negotiation.cc:240] Negotiation successful
0806 22:46:55.573143 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":164,"threads_started":1}
I20260806 22:46:55.573295 8784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:55.565813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40576 (local address 127.6.40.126:39721)
0806 22:46:55.565899 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:55.565901 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:55.566113 (+ 212us) server_negotiation.cc:407] Connection header received
0806 22:46:55.566161 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:55.566161 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:55.566172 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:55.566214 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:55.566704 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:55.567203 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:55.568021 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:55.568202 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:55.568258 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:55.570755 (+ 2497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:55.570757 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:55.570757 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:46:55.570763 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:55.572061 (+ 1298us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:55.572471 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:55.572472 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:55.572473 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:55.572527 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:55.572720 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:55.572721 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:55.572721 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:55.572821 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:55.572921 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:55.573078 (+ 157us) server_negotiation.cc:299] Negotiation successful
0806 22:46:55.573194 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":49,"server-negotiator.queue_time_us":44}
I20260806 22:46:55.575644 8715 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.6.40.126@KRBTEST.COM'} at 127.0.0.1:40576:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 22:46:55.575857 8715 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:46:55.577625 8902 tablet_service.cc:1467] Processing CreateTablet for tablet a3d5327d570546db9bfc4071972b8a73 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=75fbdc1b8cef4c8b94f9a41e5351d650]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 22:46:55.577878 8902 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3d5327d570546db9bfc4071972b8a73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:46:55.577942 9293 tablet_service.cc:1467] Processing CreateTablet for tablet a3d5327d570546db9bfc4071972b8a73 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=75fbdc1b8cef4c8b94f9a41e5351d650]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 22:46:55.577940 9098 tablet_service.cc:1467] Processing CreateTablet for tablet a3d5327d570546db9bfc4071972b8a73 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=75fbdc1b8cef4c8b94f9a41e5351d650]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 22:46:55.578122 9098 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3d5327d570546db9bfc4071972b8a73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:46:55.578121 9293 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3d5327d570546db9bfc4071972b8a73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:46:55.579579 9418 tablet_bootstrap.cc:492] T a3d5327d570546db9bfc4071972b8a73 P 4d6b767d87784ec3a9f8ebb281ae463c: Bootstrap starting.
I20260806 22:46:55.580289 9419 tablet_bootstrap.cc:492] T a3d5327d570546db9bfc4071972b8a73 P a3ae77f560314bb69116291d0eb3f635: Bootstrap starting.
I20260806 22:46:55.580668 9418 tablet_bootstrap.cc:654] T a3d5327d570546db9bfc4071972b8a73 P 4d6b767d87784ec3a9f8ebb281ae463c: Neither blocks nor log segments found. Creating new log.
I20260806 22:46:55.582026 9419 tablet_bootstrap.cc:654] T a3d5327d570546db9bfc4071972b8a73 P a3ae77f560314bb69116291d0eb3f635: Neither blocks nor log segments found. Creating new log.
I20260806 22:46:55.582088 9418 tablet_bootstrap.cc:492] T a3d5327d570546db9bfc4071972b8a73 P 4d6b767d87784ec3a9f8ebb281ae463c: No bootstrap required, opened a new log
I20260806 22:46:55.582134 9418 ts_tablet_manager.cc:1397] T a3d5327d570546db9bfc4071972b8a73 P 4d6b767d87784ec3a9f8ebb281ae463c: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260806 22:46:55.582337 9415 tablet_bootstrap.cc:492] T a3d5327d570546db9bfc4071972b8a73 P 982377b0de554faf89f0dbff3b55a8d0: Bootstrap starting.
I20260806 22:46:55.582782 9418 raft_consensus.cc:348] T a3d5327d570546db9bfc4071972b8a73 P 4d6b767d87784ec3a9f8ebb281ae463c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d6b767d87784ec3a9f8ebb281ae463c" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 33289 } } peers { permanent_uuid: "a3ae77f560314bb69116291d0eb3f635" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 46183 } } peers { permanent_uuid: "982377b0de554faf89f0dbff3b55a8d0" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 38615 } }
I20260806 22:46:55.582928 9418 raft_consensus.cc:374] T a3d5327d570546db9bfc4071972b8a73 P 4d6b767d87784ec3a9f8ebb281ae463c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:46:55.582962 9418 raft_consensus.cc:729] T a3d5327d570546db9bfc4071972b8a73 P 4d6b767d87784ec3a9f8ebb281ae463c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d6b767d87784ec3a9f8ebb281ae463c, State: Initialized, Role: FOLLOWER
I20260806 22:46:55.583027 9418 consensus_queue.cc:260] T a3d5327d570546db9bfc4071972b8a73 P 4d6b767d87784ec3a9f8ebb281ae463c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d6b767d87784ec3a9f8ebb281ae463c" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 33289 } } peers { permanent_uuid: "a3ae77f560314bb69116291d0eb3f635" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 46183 } } peers { permanent_uuid: "982377b0de554faf89f0dbff3b55a8d0" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 38615 } }
I20260806 22:46:55.583262 9418 ts_tablet_manager.cc:1428] T a3d5327d570546db9bfc4071972b8a73 P 4d6b767d87784ec3a9f8ebb281ae463c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 22:46:55.583259 9424 tablet_replica.cc:439] T a3d5327d570546db9bfc4071972b8a73 P 4d6b767d87784ec3a9f8ebb281ae463c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d6b767d87784ec3a9f8ebb281ae463c" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 33289 } } peers { permanent_uuid: "a3ae77f560314bb69116291d0eb3f635" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 46183 } } peers { permanent_uuid: "982377b0de554faf89f0dbff3b55a8d0" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 38615 } } }
I20260806 22:46:55.583402 9415 tablet_bootstrap.cc:654] T a3d5327d570546db9bfc4071972b8a73 P 982377b0de554faf89f0dbff3b55a8d0: Neither blocks nor log segments found. Creating new log.
I20260806 22:46:55.583405 9424 tablet_replica.cc:442] T a3d5327d570546db9bfc4071972b8a73 P 4d6b767d87784ec3a9f8ebb281ae463c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:46:55.583796 9419 tablet_bootstrap.cc:492] T a3d5327d570546db9bfc4071972b8a73 P a3ae77f560314bb69116291d0eb3f635: No bootstrap required, opened a new log
I20260806 22:46:55.583873 9419 ts_tablet_manager.cc:1397] T a3d5327d570546db9bfc4071972b8a73 P a3ae77f560314bb69116291d0eb3f635: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20260806 22:46:55.584342 9419 raft_consensus.cc:348] T a3d5327d570546db9bfc4071972b8a73 P a3ae77f560314bb69116291d0eb3f635 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d6b767d87784ec3a9f8ebb281ae463c" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 33289 } } peers { permanent_uuid: "a3ae77f560314bb69116291d0eb3f635" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 46183 } } peers { permanent_uuid: "982377b0de554faf89f0dbff3b55a8d0" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 38615 } }
I20260806 22:46:55.584482 9419 raft_consensus.cc:374] T a3d5327d570546db9bfc4071972b8a73 P a3ae77f560314bb69116291d0eb3f635 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:46:55.584539 9419 raft_consensus.cc:729] T a3d5327d570546db9bfc4071972b8a73 P a3ae77f560314bb69116291d0eb3f635 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3ae77f560314bb69116291d0eb3f635, State: Initialized, Role: FOLLOWER
I20260806 22:46:55.584616 9419 consensus_queue.cc:260] T a3d5327d570546db9bfc4071972b8a73 P a3ae77f560314bb69116291d0eb3f635 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d6b767d87784ec3a9f8ebb281ae463c" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 33289 } } peers { permanent_uuid: "a3ae77f560314bb69116291d0eb3f635" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 46183 } } peers { permanent_uuid: "982377b0de554faf89f0dbff3b55a8d0" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 38615 } }
I20260806 22:46:55.584858 9419 ts_tablet_manager.cc:1428] T a3d5327d570546db9bfc4071972b8a73 P a3ae77f560314bb69116291d0eb3f635: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20260806 22:46:55.585044 9415 tablet_bootstrap.cc:492] T a3d5327d570546db9bfc4071972b8a73 P 982377b0de554faf89f0dbff3b55a8d0: No bootstrap required, opened a new log
I20260806 22:46:55.584924 9423 tablet_replica.cc:439] T a3d5327d570546db9bfc4071972b8a73 P a3ae77f560314bb69116291d0eb3f635: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d6b767d87784ec3a9f8ebb281ae463c" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 33289 } } peers { permanent_uuid: "a3ae77f560314bb69116291d0eb3f635" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 46183 } } peers { permanent_uuid: "982377b0de554faf89f0dbff3b55a8d0" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 38615 } } }
I20260806 22:46:55.585101 9415 ts_tablet_manager.cc:1397] T a3d5327d570546db9bfc4071972b8a73 P 982377b0de554faf89f0dbff3b55a8d0: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 22:46:55.585104 9423 tablet_replica.cc:442] T a3d5327d570546db9bfc4071972b8a73 P a3ae77f560314bb69116291d0eb3f635: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:46:55.585569 9415 raft_consensus.cc:348] T a3d5327d570546db9bfc4071972b8a73 P 982377b0de554faf89f0dbff3b55a8d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d6b767d87784ec3a9f8ebb281ae463c" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 33289 } } peers { permanent_uuid: "a3ae77f560314bb69116291d0eb3f635" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 46183 } } peers { permanent_uuid: "982377b0de554faf89f0dbff3b55a8d0" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 38615 } }
I20260806 22:46:55.585672 9415 raft_consensus.cc:374] T a3d5327d570546db9bfc4071972b8a73 P 982377b0de554faf89f0dbff3b55a8d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:46:55.585700 9415 raft_consensus.cc:729] T a3d5327d570546db9bfc4071972b8a73 P 982377b0de554faf89f0dbff3b55a8d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 982377b0de554faf89f0dbff3b55a8d0, State: Initialized, Role: FOLLOWER
I20260806 22:46:55.585750 9415 consensus_queue.cc:260] T a3d5327d570546db9bfc4071972b8a73 P 982377b0de554faf89f0dbff3b55a8d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d6b767d87784ec3a9f8ebb281ae463c" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 33289 } } peers { permanent_uuid: "a3ae77f560314bb69116291d0eb3f635" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 46183 } } peers { permanent_uuid: "982377b0de554faf89f0dbff3b55a8d0" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 38615 } }
I20260806 22:46:55.585906 9415 ts_tablet_manager.cc:1428] T a3d5327d570546db9bfc4071972b8a73 P 982377b0de554faf89f0dbff3b55a8d0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 22:46:55.585980 9420 tablet_replica.cc:439] T a3d5327d570546db9bfc4071972b8a73 P 982377b0de554faf89f0dbff3b55a8d0: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d6b767d87784ec3a9f8ebb281ae463c" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 33289 } } peers { permanent_uuid: "a3ae77f560314bb69116291d0eb3f635" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 46183 } } peers { permanent_uuid: "982377b0de554faf89f0dbff3b55a8d0" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 38615 } } }
I20260806 22:46:55.586099 9420 tablet_replica.cc:442] T a3d5327d570546db9bfc4071972b8a73 P 982377b0de554faf89f0dbff3b55a8d0: This TxnStatusTablet replica's current role is: FOLLOWER
W20260806 22:46:55.610083 9388 tablet.cc:2369] T a3d5327d570546db9bfc4071972b8a73 P 982377b0de554faf89f0dbff3b55a8d0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:46:55.669713 9424 raft_consensus.cc:482] T a3d5327d570546db9bfc4071972b8a73 P 4d6b767d87784ec3a9f8ebb281ae463c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:46:55.669843 9424 raft_consensus.cc:504] T a3d5327d570546db9bfc4071972b8a73 P 4d6b767d87784ec3a9f8ebb281ae463c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d6b767d87784ec3a9f8ebb281ae463c" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 33289 } } peers { permanent_uuid: "a3ae77f560314bb69116291d0eb3f635" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 46183 } } peers { permanent_uuid: "982377b0de554faf89f0dbff3b55a8d0" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 38615 } }
I20260806 22:46:55.670318 9424 leader_election.cc:288] T a3d5327d570546db9bfc4071972b8a73 P 4d6b767d87784ec3a9f8ebb281ae463c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3ae77f560314bb69116291d0eb3f635 (127.6.40.66:46183), 982377b0de554faf89f0dbff3b55a8d0 (127.6.40.67:38615)
I20260806 22:46:55.675052 9414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:55.670750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.65:59229 (local address 127.6.40.66:46183)
0806 22:46:55.670821 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:55.670824 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:55.670834 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:46:55.670866 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:55.670867 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:55.670878 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:55.670942 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:46:55.671206 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:55.671994 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:55.673852 (+ 1858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:55.674454 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:55.674541 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:55.674557 (+ 16us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:55.674947 (+ 390us) server_negotiation.cc:299] Negotiation successful
0806 22:46:55.674991 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 22:46:55.675241 9442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:55.670517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.66:46183 (local address 127.6.40.65:59229)
0806 22:46:55.670711 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:55.670722 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:55.670785 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:55.670961 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:55.670961 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:55.670964 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:46:55.671105 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:55.671109 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:55.672114 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:55.672115 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:55.673757 (+ 1642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:55.673761 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:55.674734 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:55.674737 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:55.674848 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:55.674855 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 22:46:55.674931 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 22:46:55.674998 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"mutex_wait_us":86,"thread_start_us":105,"threads_started":1}
I20260806 22:46:55.675504 9411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:55.670650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.65:35411 (local address 127.6.40.67:38615)
0806 22:46:55.670704 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:55.670706 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:55.670911 (+ 205us) server_negotiation.cc:407] Connection header received
0806 22:46:55.671034 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:55.671035 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:55.671049 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:55.671139 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:46:55.671408 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:55.672159 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:55.673851 (+ 1692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:55.674508 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:55.674575 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:55.674585 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:55.675347 (+ 762us) server_negotiation.cc:299] Negotiation successful
0806 22:46:55.675403 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 22:46:55.675565 9125 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3d5327d570546db9bfc4071972b8a73" candidate_uuid: "4d6b767d87784ec3a9f8ebb281ae463c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3ae77f560314bb69116291d0eb3f635" is_pre_election: true
I20260806 22:46:55.675671 9125 raft_consensus.cc:2454] T a3d5327d570546db9bfc4071972b8a73 P a3ae77f560314bb69116291d0eb3f635 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d6b767d87784ec3a9f8ebb281ae463c in term 0.
I20260806 22:46:55.675799 9443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:55.670517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.67:38615 (local address 127.6.40.65:35411)
0806 22:46:55.670834 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:55.670846 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:55.670918 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:55.671194 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:55.671195 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:55.671197 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:46:55.671293 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:55.671295 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:55.672258 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:55.672259 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:55.673758 (+ 1499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:55.673761 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:55.675109 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:55.675111 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:55.675227 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:55.675240 (+ 13us) client_negotiation.cc:769] Sending connection context
0806 22:46:55.675593 (+ 353us) client_negotiation.cc:240] Negotiation successful
0806 22:46:55.675647 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"mutex_wait_us":44,"spinlock_wait_cycles":14080,"thread_start_us":101,"threads_started":1}
I20260806 22:46:55.675911 8838 leader_election.cc:302] T a3d5327d570546db9bfc4071972b8a73 P 4d6b767d87784ec3a9f8ebb281ae463c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d6b767d87784ec3a9f8ebb281ae463c, a3ae77f560314bb69116291d0eb3f635; no voters:
I20260806 22:46:55.676059 9424 raft_consensus.cc:2781] T a3d5327d570546db9bfc4071972b8a73 P 4d6b767d87784ec3a9f8ebb281ae463c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:46:55.676105 9424 raft_consensus.cc:482] T a3d5327d570546db9bfc4071972b8a73 P 4d6b767d87784ec3a9f8ebb281ae463c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:46:55.676129 9424 raft_consensus.cc:3037] T a3d5327d570546db9bfc4071972b8a73 P 4d6b767d87784ec3a9f8ebb281ae463c [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:46:55.676438 9320 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3d5327d570546db9bfc4071972b8a73" candidate_uuid: "4d6b767d87784ec3a9f8ebb281ae463c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "982377b0de554faf89f0dbff3b55a8d0" is_pre_election: true
I20260806 22:46:55.676579 9320 raft_consensus.cc:2454] T a3d5327d570546db9bfc4071972b8a73 P 982377b0de554faf89f0dbff3b55a8d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d6b767d87784ec3a9f8ebb281ae463c in term 0.
I20260806 22:46:55.677403 9424 raft_consensus.cc:504] T a3d5327d570546db9bfc4071972b8a73 P 4d6b767d87784ec3a9f8ebb281ae463c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d6b767d87784ec3a9f8ebb281ae463c" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 33289 } } peers { permanent_uuid: "a3ae77f560314bb69116291d0eb3f635" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 46183 } } peers { permanent_uuid: "982377b0de554faf89f0dbff3b55a8d0" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 38615 } }
I20260806 22:46:55.677562 9424 leader_election.cc:288] T a3d5327d570546db9bfc4071972b8a73 P 4d6b767d87784ec3a9f8ebb281ae463c [CANDIDATE]: Term 1 election: Requested vote from peers a3ae77f560314bb69116291d0eb3f635 (127.6.40.66:46183), 982377b0de554faf89f0dbff3b55a8d0 (127.6.40.67:38615)
I20260806 22:46:55.677743 9125 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3d5327d570546db9bfc4071972b8a73" candidate_uuid: "4d6b767d87784ec3a9f8ebb281ae463c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3ae77f560314bb69116291d0eb3f635"
I20260806 22:46:55.677815 9125 raft_consensus.cc:3037] T a3d5327d570546db9bfc4071972b8a73 P a3ae77f560314bb69116291d0eb3f635 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:46:55.678750 9125 raft_consensus.cc:2454] T a3d5327d570546db9bfc4071972b8a73 P a3ae77f560314bb69116291d0eb3f635 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d6b767d87784ec3a9f8ebb281ae463c in term 1.
I20260806 22:46:55.678887 9320 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3d5327d570546db9bfc4071972b8a73" candidate_uuid: "4d6b767d87784ec3a9f8ebb281ae463c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "982377b0de554faf89f0dbff3b55a8d0"
I20260806 22:46:55.678938 8838 leader_election.cc:302] T a3d5327d570546db9bfc4071972b8a73 P 4d6b767d87784ec3a9f8ebb281ae463c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d6b767d87784ec3a9f8ebb281ae463c, a3ae77f560314bb69116291d0eb3f635; no voters:
I20260806 22:46:55.678982 9320 raft_consensus.cc:3037] T a3d5327d570546db9bfc4071972b8a73 P 982377b0de554faf89f0dbff3b55a8d0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:46:55.679064 9424 raft_consensus.cc:2781] T a3d5327d570546db9bfc4071972b8a73 P 4d6b767d87784ec3a9f8ebb281ae463c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:46:55.679216 9424 raft_consensus.cc:686] T a3d5327d570546db9bfc4071972b8a73 P 4d6b767d87784ec3a9f8ebb281ae463c [term 1 LEADER]: Becoming Leader. State: Replica: 4d6b767d87784ec3a9f8ebb281ae463c, State: Running, Role: LEADER
I20260806 22:46:55.679312 9424 consensus_queue.cc:237] T a3d5327d570546db9bfc4071972b8a73 P 4d6b767d87784ec3a9f8ebb281ae463c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d6b767d87784ec3a9f8ebb281ae463c" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 33289 } } peers { permanent_uuid: "a3ae77f560314bb69116291d0eb3f635" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 46183 } } peers { permanent_uuid: "982377b0de554faf89f0dbff3b55a8d0" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 38615 } }
I20260806 22:46:55.680089 9444 tablet_replica.cc:439] T a3d5327d570546db9bfc4071972b8a73 P 4d6b767d87784ec3a9f8ebb281ae463c: TxnStatusTablet state changed. Reason: New leader 4d6b767d87784ec3a9f8ebb281ae463c. Latest consensus state: current_term: 1 leader_uuid: "4d6b767d87784ec3a9f8ebb281ae463c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d6b767d87784ec3a9f8ebb281ae463c" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 33289 } } peers { permanent_uuid: "a3ae77f560314bb69116291d0eb3f635" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 46183 } } peers { permanent_uuid: "982377b0de554faf89f0dbff3b55a8d0" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 38615 } } }
I20260806 22:46:55.680238 9444 tablet_replica.cc:442] T a3d5327d570546db9bfc4071972b8a73 P 4d6b767d87784ec3a9f8ebb281ae463c: This TxnStatusTablet replica's current role is: LEADER
I20260806 22:46:55.680371 9320 raft_consensus.cc:2454] T a3d5327d570546db9bfc4071972b8a73 P 982377b0de554faf89f0dbff3b55a8d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d6b767d87784ec3a9f8ebb281ae463c in term 1.
I20260806 22:46:55.680332 8716 catalog_manager.cc:5495] T a3d5327d570546db9bfc4071972b8a73 P 4d6b767d87784ec3a9f8ebb281ae463c reported cstate change: term changed from 0 to 1, leader changed from <none> to 4d6b767d87784ec3a9f8ebb281ae463c (127.6.40.65). New cstate: current_term: 1 leader_uuid: "4d6b767d87784ec3a9f8ebb281ae463c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d6b767d87784ec3a9f8ebb281ae463c" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 33289 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a3ae77f560314bb69116291d0eb3f635" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 46183 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "982377b0de554faf89f0dbff3b55a8d0" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 38615 } health_report { overall_health: UNKNOWN } } }
I20260806 22:46:55.680559 9448 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 22:46:55.680668 9448 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 22:46:55.702932 9412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:55.699982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33806 (local address 127.6.40.65:33289)
0806 22:46:55.700036 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:55.700039 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:55.700049 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:46:55.700127 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:55.700128 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:55.700154 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:55.700234 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:46:55.700389 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:55.701169 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:55.702019 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:55.702228 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:55.702316 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:55.702542 (+ 226us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:46:55.702658 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:46:55.702756 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:55.702811 (+ 55us) server_negotiation.cc:299] Negotiation successful
0806 22:46:55.702839 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 22:46:55.702944 9435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:55.699866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.65:33289 (local address 127.0.0.1:33806)
0806 22:46:55.699963 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:55.699976 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:55.700030 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:55.700244 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:55.700245 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:55.700246 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 22:46:55.700307 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:55.700310 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:55.701290 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:55.701292 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:55.701900 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:55.701903 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:55.702319 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:55.702320 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:55.702396 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:55.702422 (+ 26us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 22:46:55.702733 (+ 311us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 22:46:55.702747 (+ 14us) client_negotiation.cc:769] Sending connection context
0806 22:46:55.702804 (+ 57us) client_negotiation.cc:240] Negotiation successful
0806 22:46:55.702864 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260806 22:46:55.705621 9125 raft_consensus.cc:1261] T a3d5327d570546db9bfc4071972b8a73 P a3ae77f560314bb69116291d0eb3f635 [term 1 FOLLOWER]: Refusing update from remote peer 4d6b767d87784ec3a9f8ebb281ae463c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:46:55.705770 9320 raft_consensus.cc:1261] T a3d5327d570546db9bfc4071972b8a73 P 982377b0de554faf89f0dbff3b55a8d0 [term 1 FOLLOWER]: Refusing update from remote peer 4d6b767d87784ec3a9f8ebb281ae463c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:46:55.706025 9424 consensus_queue.cc:1035] T a3d5327d570546db9bfc4071972b8a73 P 4d6b767d87784ec3a9f8ebb281ae463c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3ae77f560314bb69116291d0eb3f635" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 46183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:46:55.706189 9444 consensus_queue.cc:1035] T a3d5327d570546db9bfc4071972b8a73 P 4d6b767d87784ec3a9f8ebb281ae463c [LEADER]: Connected to new peer: Peer: permanent_uuid: "982377b0de554faf89f0dbff3b55a8d0" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 38615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:46:55.707523 9423 tablet_replica.cc:439] T a3d5327d570546db9bfc4071972b8a73 P a3ae77f560314bb69116291d0eb3f635: TxnStatusTablet state changed. Reason: New leader 4d6b767d87784ec3a9f8ebb281ae463c. Latest consensus state: current_term: 1 leader_uuid: "4d6b767d87784ec3a9f8ebb281ae463c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d6b767d87784ec3a9f8ebb281ae463c" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 33289 } } peers { permanent_uuid: "a3ae77f560314bb69116291d0eb3f635" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 46183 } } peers { permanent_uuid: "982377b0de554faf89f0dbff3b55a8d0" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 38615 } } }
I20260806 22:46:55.707626 9423 tablet_replica.cc:442] T a3d5327d570546db9bfc4071972b8a73 P a3ae77f560314bb69116291d0eb3f635: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:46:55.708067 9420 tablet_replica.cc:439] T a3d5327d570546db9bfc4071972b8a73 P 982377b0de554faf89f0dbff3b55a8d0: TxnStatusTablet state changed. Reason: New leader 4d6b767d87784ec3a9f8ebb281ae463c. Latest consensus state: current_term: 1 leader_uuid: "4d6b767d87784ec3a9f8ebb281ae463c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d6b767d87784ec3a9f8ebb281ae463c" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 33289 } } peers { permanent_uuid: "a3ae77f560314bb69116291d0eb3f635" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 46183 } } peers { permanent_uuid: "982377b0de554faf89f0dbff3b55a8d0" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 38615 } } }
I20260806 22:46:55.708173 9420 tablet_replica.cc:442] T a3d5327d570546db9bfc4071972b8a73 P 982377b0de554faf89f0dbff3b55a8d0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:46:55.709069 9444 tablet_replica.cc:439] T a3d5327d570546db9bfc4071972b8a73 P 4d6b767d87784ec3a9f8ebb281ae463c: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4d6b767d87784ec3a9f8ebb281ae463c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d6b767d87784ec3a9f8ebb281ae463c" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 33289 } } peers { permanent_uuid: "a3ae77f560314bb69116291d0eb3f635" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 46183 } } peers { permanent_uuid: "982377b0de554faf89f0dbff3b55a8d0" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 38615 } } }
I20260806 22:46:55.709160 9444 tablet_replica.cc:442] T a3d5327d570546db9bfc4071972b8a73 P 4d6b767d87784ec3a9f8ebb281ae463c: This TxnStatusTablet replica's current role is: LEADER
I20260806 22:46:55.709138 9423 tablet_replica.cc:439] T a3d5327d570546db9bfc4071972b8a73 P a3ae77f560314bb69116291d0eb3f635: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4d6b767d87784ec3a9f8ebb281ae463c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d6b767d87784ec3a9f8ebb281ae463c" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 33289 } } peers { permanent_uuid: "a3ae77f560314bb69116291d0eb3f635" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 46183 } } peers { permanent_uuid: "982377b0de554faf89f0dbff3b55a8d0" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 38615 } } }
I20260806 22:46:55.709230 9423 tablet_replica.cc:442] T a3d5327d570546db9bfc4071972b8a73 P a3ae77f560314bb69116291d0eb3f635: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:46:55.709293 9451 mvcc.cc:205] Tried to move back new op lower bound from 7315687078726627328 to 7315687078622998528. Current Snapshot: MvccSnapshot[applied={T|T < 7315687078726627328}]
I20260806 22:46:55.710263 9424 tablet_replica.cc:439] T a3d5327d570546db9bfc4071972b8a73 P 4d6b767d87784ec3a9f8ebb281ae463c: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4d6b767d87784ec3a9f8ebb281ae463c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d6b767d87784ec3a9f8ebb281ae463c" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 33289 } } peers { permanent_uuid: "a3ae77f560314bb69116291d0eb3f635" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 46183 } } peers { permanent_uuid: "982377b0de554faf89f0dbff3b55a8d0" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 38615 } } }
I20260806 22:46:55.710351 9424 tablet_replica.cc:442] T a3d5327d570546db9bfc4071972b8a73 P 4d6b767d87784ec3a9f8ebb281ae463c: This TxnStatusTablet replica's current role is: LEADER
I20260806 22:46:55.710915 9420 tablet_replica.cc:439] T a3d5327d570546db9bfc4071972b8a73 P 982377b0de554faf89f0dbff3b55a8d0: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4d6b767d87784ec3a9f8ebb281ae463c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d6b767d87784ec3a9f8ebb281ae463c" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 33289 } } peers { permanent_uuid: "a3ae77f560314bb69116291d0eb3f635" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 46183 } } peers { permanent_uuid: "982377b0de554faf89f0dbff3b55a8d0" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 38615 } } }
I20260806 22:46:55.711004 9420 tablet_replica.cc:442] T a3d5327d570546db9bfc4071972b8a73 P 982377b0de554faf89f0dbff3b55a8d0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:46:55.714666 9411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:55.711948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57554 (local address 127.6.40.67:38615)
0806 22:46:55.711988 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:55.711990 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:55.711999 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:46:55.712033 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:55.712034 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:55.712055 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:55.712139 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:46:55.712330 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:55.713025 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:55.713821 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:55.714040 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:55.714117 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:55.714329 (+ 212us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:46:55.714443 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:46:55.714535 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:55.714559 (+ 24us) server_negotiation.cc:299] Negotiation successful
0806 22:46:55.714583 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 22:46:55.720993 9412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:55.717964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.67:34211 (local address 127.6.40.65:33289)
0806 22:46:55.718067 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:55.718069 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:55.718080 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:46:55.718105 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:55.718106 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:55.718198 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:55.718274 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:46:55.718522 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:55.719086 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:55.720124 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:55.720320 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:55.720429 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:55.720707 (+ 278us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:46:55.720738 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:46:55.720802 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:55.720898 (+ 96us) server_negotiation.cc:299] Negotiation successful
0806 22:46:55.720919 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 22:46:55.721011 9395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:55.717870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.65:33289 (local address 127.6.40.67:34211)
0806 22:46:55.717966 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:55.717978 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:55.718031 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:55.718282 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:55.718282 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:55.718284 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 22:46:55.718436 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:55.718438 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:55.719170 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:55.719171 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:55.720034 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:55.720036 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:55.720428 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:55.720429 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:55.720548 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:55.720564 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 22:46:55.720804 (+ 240us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 22:46:55.720807 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:46:55.720893 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 22:46:55.720946 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260806 22:46:55.723722 9125 raft_consensus.cc:1261] T fd91884bdffe4d3685831649a184aa1a P a3ae77f560314bb69116291d0eb3f635 [term 1 FOLLOWER]: Refusing update from remote peer 982377b0de554faf89f0dbff3b55a8d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:46:55.724062 8933 raft_consensus.cc:1261] T fd91884bdffe4d3685831649a184aa1a P 4d6b767d87784ec3a9f8ebb281ae463c [term 1 FOLLOWER]: Refusing update from remote peer 982377b0de554faf89f0dbff3b55a8d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:46:55.724102 9426 consensus_queue.cc:1035] T fd91884bdffe4d3685831649a184aa1a P 982377b0de554faf89f0dbff3b55a8d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3ae77f560314bb69116291d0eb3f635" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 46183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:46:55.724377 9426 consensus_queue.cc:1035] T fd91884bdffe4d3685831649a184aa1a P 982377b0de554faf89f0dbff3b55a8d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d6b767d87784ec3a9f8ebb281ae463c" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 33289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:46:55.725920 9452 mvcc.cc:205] Tried to move back new op lower bound from 7315687078802468864 to 7315687077975101440. Current Snapshot: MvccSnapshot[applied={T|T < 7315687078802468864}]
I20260806 22:46:55.736028 9411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:55.732110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.65:50531 (local address 127.6.40.67:38615)
0806 22:46:55.732146 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:55.732148 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:55.732490 (+ 342us) server_negotiation.cc:407] Connection header received
0806 22:46:55.732560 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:55.732561 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:55.732574 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:55.732630 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:46:55.732933 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:55.733580 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:55.735177 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:55.735360 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:55.735430 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:55.735687 (+ 257us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:46:55.735748 (+ 61us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:46:55.735782 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:55.735926 (+ 144us) server_negotiation.cc:299] Negotiation successful
0806 22:46:55.735950 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260806 22:46:55.736155 9469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:55.732058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.67:38615 (local address 127.6.40.65:50531)
0806 22:46:55.732365 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:55.732376 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:55.732509 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:55.732634 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:55.732634 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:55.732636 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 22:46:55.732823 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:55.732826 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:55.733715 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:55.733715 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:55.735071 (+ 1356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:55.735074 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:55.735442 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:55.735443 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:55.735556 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:55.735574 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 22:46:55.735828 (+ 254us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 22:46:55.735831 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:46:55.735875 (+ 44us) client_negotiation.cc:240] Negotiation successful
0806 22:46:55.735928 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":146,"threads_started":1}
I20260806 22:46:55.748535 8715 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40564:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:46:55.748689 8715 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40564:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:46:55.749689 8715 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9d0e72021aa443289c0bb45de789d382: Sending DeleteTablet for 3 replicas of tablet fd91884bdffe4d3685831649a184aa1a
I20260806 22:46:55.750077 8902 tablet_service.cc:1514] Processing DeleteTablet for tablet fd91884bdffe4d3685831649a184aa1a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:46:55 UTC) from {username='kudu', principal='kudu/127.6.40.126@KRBTEST.COM'} at 127.0.0.1:33792
I20260806 22:46:55.750162 9098 tablet_service.cc:1514] Processing DeleteTablet for tablet fd91884bdffe4d3685831649a184aa1a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:46:55 UTC) from {username='kudu', principal='kudu/127.6.40.126@KRBTEST.COM'} at 127.0.0.1:51600
I20260806 22:46:55.750543 9293 tablet_service.cc:1514] Processing DeleteTablet for tablet fd91884bdffe4d3685831649a184aa1a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:46:55 UTC) from {username='kudu', principal='kudu/127.6.40.126@KRBTEST.COM'} at 127.0.0.1:57548
I20260806 22:46:55.750674 9476 tablet_replica.cc:330] T fd91884bdffe4d3685831649a184aa1a P 4d6b767d87784ec3a9f8ebb281ae463c: stopping tablet replica
I20260806 22:46:55.750926 9476 raft_consensus.cc:2229] T fd91884bdffe4d3685831649a184aa1a P 4d6b767d87784ec3a9f8ebb281ae463c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:46:55.751075 9476 raft_consensus.cc:2258] T fd91884bdffe4d3685831649a184aa1a P 4d6b767d87784ec3a9f8ebb281ae463c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:46:55.751101 8715 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40564:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:46:55.751292 8715 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:46:55.751896 9476 ts_tablet_manager.cc:1905] T fd91884bdffe4d3685831649a184aa1a P 4d6b767d87784ec3a9f8ebb281ae463c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:46:55.752913 8902 tablet_service.cc:1467] Processing CreateTablet for tablet d3b81a69eadd4ec294fad3662707cf7c (DEFAULT_TABLE table=test-table [id=962c13b0eb904d6cab92a0c8c06c62f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:46:55.753077 8902 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3b81a69eadd4ec294fad3662707cf7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:46:55.755000 9476 ts_tablet_manager.cc:1918] T fd91884bdffe4d3685831649a184aa1a P 4d6b767d87784ec3a9f8ebb281ae463c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:46:55.755069 9476 log.cc:1196] T fd91884bdffe4d3685831649a184aa1a P 4d6b767d87784ec3a9f8ebb281ae463c: Deleting WAL directory at /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/minicluster-data/ts-0/wal/wals/fd91884bdffe4d3685831649a184aa1a
I20260806 22:46:55.755244 9418 tablet_bootstrap.cc:492] T d3b81a69eadd4ec294fad3662707cf7c P 4d6b767d87784ec3a9f8ebb281ae463c: Bootstrap starting.
I20260806 22:46:55.755427 9476 ts_tablet_manager.cc:1939] T fd91884bdffe4d3685831649a184aa1a P 4d6b767d87784ec3a9f8ebb281ae463c: Deleting consensus metadata
I20260806 22:46:55.755484 9098 tablet_service.cc:1467] Processing CreateTablet for tablet d3b81a69eadd4ec294fad3662707cf7c (DEFAULT_TABLE table=test-table [id=962c13b0eb904d6cab92a0c8c06c62f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:46:55.755695 9098 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3b81a69eadd4ec294fad3662707cf7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:46:55.755836 9478 tablet_replica.cc:330] T fd91884bdffe4d3685831649a184aa1a P 982377b0de554faf89f0dbff3b55a8d0: stopping tablet replica
I20260806 22:46:55.755959 9478 raft_consensus.cc:2229] T fd91884bdffe4d3685831649a184aa1a P 982377b0de554faf89f0dbff3b55a8d0 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:46:55.756063 9478 raft_consensus.cc:2258] T fd91884bdffe4d3685831649a184aa1a P 982377b0de554faf89f0dbff3b55a8d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:46:55.756224 9477 tablet_replica.cc:330] T fd91884bdffe4d3685831649a184aa1a P a3ae77f560314bb69116291d0eb3f635: stopping tablet replica
I20260806 22:46:55.756342 9477 raft_consensus.cc:2229] T fd91884bdffe4d3685831649a184aa1a P a3ae77f560314bb69116291d0eb3f635 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:46:55.756361 9418 tablet_bootstrap.cc:654] T d3b81a69eadd4ec294fad3662707cf7c P 4d6b767d87784ec3a9f8ebb281ae463c: Neither blocks nor log segments found. Creating new log.
I20260806 22:46:55.756419 9477 raft_consensus.cc:2258] T fd91884bdffe4d3685831649a184aa1a P a3ae77f560314bb69116291d0eb3f635 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:46:55.757467 8698 catalog_manager.cc:4841] TS 4d6b767d87784ec3a9f8ebb281ae463c (127.6.40.65:33289): tablet fd91884bdffe4d3685831649a184aa1a (table test-table [id=1db510231e024950a296d324388d16f7]) successfully deleted
I20260806 22:46:55.758111 9419 tablet_bootstrap.cc:492] T d3b81a69eadd4ec294fad3662707cf7c P a3ae77f560314bb69116291d0eb3f635: Bootstrap starting.
I20260806 22:46:55.758359 9293 tablet_service.cc:1467] Processing CreateTablet for tablet d3b81a69eadd4ec294fad3662707cf7c (DEFAULT_TABLE table=test-table [id=962c13b0eb904d6cab92a0c8c06c62f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:46:55.758495 9293 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3b81a69eadd4ec294fad3662707cf7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:46:55.759099 9419 tablet_bootstrap.cc:654] T d3b81a69eadd4ec294fad3662707cf7c P a3ae77f560314bb69116291d0eb3f635: Neither blocks nor log segments found. Creating new log.
I20260806 22:46:55.760288 9415 tablet_bootstrap.cc:492] T d3b81a69eadd4ec294fad3662707cf7c P 982377b0de554faf89f0dbff3b55a8d0: Bootstrap starting.
I20260806 22:46:55.760859 9419 tablet_bootstrap.cc:492] T d3b81a69eadd4ec294fad3662707cf7c P a3ae77f560314bb69116291d0eb3f635: No bootstrap required, opened a new log
I20260806 22:46:55.760910 9419 ts_tablet_manager.cc:1397] T d3b81a69eadd4ec294fad3662707cf7c P a3ae77f560314bb69116291d0eb3f635: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 22:46:55.761302 9419 raft_consensus.cc:348] T d3b81a69eadd4ec294fad3662707cf7c P a3ae77f560314bb69116291d0eb3f635 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d6b767d87784ec3a9f8ebb281ae463c" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 33289 } } peers { permanent_uuid: "a3ae77f560314bb69116291d0eb3f635" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 46183 } } peers { permanent_uuid: "982377b0de554faf89f0dbff3b55a8d0" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 38615 } }
I20260806 22:46:55.761428 9419 raft_consensus.cc:374] T d3b81a69eadd4ec294fad3662707cf7c P a3ae77f560314bb69116291d0eb3f635 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:46:55.761456 9419 raft_consensus.cc:729] T d3b81a69eadd4ec294fad3662707cf7c P a3ae77f560314bb69116291d0eb3f635 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3ae77f560314bb69116291d0eb3f635, State: Initialized, Role: FOLLOWER
I20260806 22:46:55.761549 9415 tablet_bootstrap.cc:654] T d3b81a69eadd4ec294fad3662707cf7c P 982377b0de554faf89f0dbff3b55a8d0: Neither blocks nor log segments found. Creating new log.
I20260806 22:46:55.761554 9419 consensus_queue.cc:260] T d3b81a69eadd4ec294fad3662707cf7c P a3ae77f560314bb69116291d0eb3f635 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d6b767d87784ec3a9f8ebb281ae463c" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 33289 } } peers { permanent_uuid: "a3ae77f560314bb69116291d0eb3f635" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 46183 } } peers { permanent_uuid: "982377b0de554faf89f0dbff3b55a8d0" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 38615 } }
I20260806 22:46:55.761686 9419 ts_tablet_manager.cc:1428] T d3b81a69eadd4ec294fad3662707cf7c P a3ae77f560314bb69116291d0eb3f635: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 22:46:55.762998 9415 tablet_bootstrap.cc:492] T d3b81a69eadd4ec294fad3662707cf7c P 982377b0de554faf89f0dbff3b55a8d0: No bootstrap required, opened a new log
I20260806 22:46:55.763057 9415 ts_tablet_manager.cc:1397] T d3b81a69eadd4ec294fad3662707cf7c P 982377b0de554faf89f0dbff3b55a8d0: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260806 22:46:55.763506 9415 raft_consensus.cc:348] T d3b81a69eadd4ec294fad3662707cf7c P 982377b0de554faf89f0dbff3b55a8d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d6b767d87784ec3a9f8ebb281ae463c" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 33289 } } peers { permanent_uuid: "a3ae77f560314bb69116291d0eb3f635" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 46183 } } peers { permanent_uuid: "982377b0de554faf89f0dbff3b55a8d0" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 38615 } }
I20260806 22:46:55.763597 9415 raft_consensus.cc:374] T d3b81a69eadd4ec294fad3662707cf7c P 982377b0de554faf89f0dbff3b55a8d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:46:55.763622 9415 raft_consensus.cc:729] T d3b81a69eadd4ec294fad3662707cf7c P 982377b0de554faf89f0dbff3b55a8d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 982377b0de554faf89f0dbff3b55a8d0, State: Initialized, Role: FOLLOWER
I20260806 22:46:55.763664 9415 consensus_queue.cc:260] T d3b81a69eadd4ec294fad3662707cf7c P 982377b0de554faf89f0dbff3b55a8d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d6b767d87784ec3a9f8ebb281ae463c" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 33289 } } peers { permanent_uuid: "a3ae77f560314bb69116291d0eb3f635" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 46183 } } peers { permanent_uuid: "982377b0de554faf89f0dbff3b55a8d0" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 38615 } }
I20260806 22:46:55.764038 9478 ts_tablet_manager.cc:1905] T fd91884bdffe4d3685831649a184aa1a P 982377b0de554faf89f0dbff3b55a8d0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:46:55.764567 9415 ts_tablet_manager.cc:1428] T d3b81a69eadd4ec294fad3662707cf7c P 982377b0de554faf89f0dbff3b55a8d0: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260806 22:46:55.766386 9477 ts_tablet_manager.cc:1905] T fd91884bdffe4d3685831649a184aa1a P a3ae77f560314bb69116291d0eb3f635: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:46:55.768291 9477 ts_tablet_manager.cc:1918] T fd91884bdffe4d3685831649a184aa1a P a3ae77f560314bb69116291d0eb3f635: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:46:55.768373 9477 log.cc:1196] T fd91884bdffe4d3685831649a184aa1a P a3ae77f560314bb69116291d0eb3f635: Deleting WAL directory at /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/minicluster-data/ts-1/wal/wals/fd91884bdffe4d3685831649a184aa1a
I20260806 22:46:55.768635 9477 ts_tablet_manager.cc:1939] T fd91884bdffe4d3685831649a184aa1a P a3ae77f560314bb69116291d0eb3f635: Deleting consensus metadata
I20260806 22:46:55.769155 8696 catalog_manager.cc:4841] TS a3ae77f560314bb69116291d0eb3f635 (127.6.40.66:46183): tablet fd91884bdffe4d3685831649a184aa1a (table test-table [id=1db510231e024950a296d324388d16f7]) successfully deleted
I20260806 22:46:55.769972 9478 ts_tablet_manager.cc:1918] T fd91884bdffe4d3685831649a184aa1a P 982377b0de554faf89f0dbff3b55a8d0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:46:55.770053 9478 log.cc:1196] T fd91884bdffe4d3685831649a184aa1a P 982377b0de554faf89f0dbff3b55a8d0: Deleting WAL directory at /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/minicluster-data/ts-2/wal/wals/fd91884bdffe4d3685831649a184aa1a
I20260806 22:46:55.770314 9478 ts_tablet_manager.cc:1939] T fd91884bdffe4d3685831649a184aa1a P 982377b0de554faf89f0dbff3b55a8d0: Deleting consensus metadata
I20260806 22:46:55.770807 8697 catalog_manager.cc:4841] TS 982377b0de554faf89f0dbff3b55a8d0 (127.6.40.67:38615): tablet fd91884bdffe4d3685831649a184aa1a (table test-table [id=1db510231e024950a296d324388d16f7]) successfully deleted
I20260806 22:46:55.773406 9418 tablet_bootstrap.cc:492] T d3b81a69eadd4ec294fad3662707cf7c P 4d6b767d87784ec3a9f8ebb281ae463c: No bootstrap required, opened a new log
I20260806 22:46:55.773471 9418 ts_tablet_manager.cc:1397] T d3b81a69eadd4ec294fad3662707cf7c P 4d6b767d87784ec3a9f8ebb281ae463c: Time spent bootstrapping tablet: real 0.018s user 0.000s sys 0.002s
I20260806 22:46:55.773715 9418 raft_consensus.cc:348] T d3b81a69eadd4ec294fad3662707cf7c P 4d6b767d87784ec3a9f8ebb281ae463c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d6b767d87784ec3a9f8ebb281ae463c" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 33289 } } peers { permanent_uuid: "a3ae77f560314bb69116291d0eb3f635" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 46183 } } peers { permanent_uuid: "982377b0de554faf89f0dbff3b55a8d0" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 38615 } }
I20260806 22:46:55.773795 9418 raft_consensus.cc:374] T d3b81a69eadd4ec294fad3662707cf7c P 4d6b767d87784ec3a9f8ebb281ae463c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:46:55.773815 9418 raft_consensus.cc:729] T d3b81a69eadd4ec294fad3662707cf7c P 4d6b767d87784ec3a9f8ebb281ae463c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d6b767d87784ec3a9f8ebb281ae463c, State: Initialized, Role: FOLLOWER
I20260806 22:46:55.773866 9418 consensus_queue.cc:260] T d3b81a69eadd4ec294fad3662707cf7c P 4d6b767d87784ec3a9f8ebb281ae463c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d6b767d87784ec3a9f8ebb281ae463c" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 33289 } } peers { permanent_uuid: "a3ae77f560314bb69116291d0eb3f635" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 46183 } } peers { permanent_uuid: "982377b0de554faf89f0dbff3b55a8d0" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 38615 } }
I20260806 22:46:55.774030 9418 ts_tablet_manager.cc:1428] T d3b81a69eadd4ec294fad3662707cf7c P 4d6b767d87784ec3a9f8ebb281ae463c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 22:46:55.820381 9471 raft_consensus.cc:482] T d3b81a69eadd4ec294fad3662707cf7c P 4d6b767d87784ec3a9f8ebb281ae463c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:46:55.820497 9471 raft_consensus.cc:504] T d3b81a69eadd4ec294fad3662707cf7c P 4d6b767d87784ec3a9f8ebb281ae463c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d6b767d87784ec3a9f8ebb281ae463c" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 33289 } } peers { permanent_uuid: "a3ae77f560314bb69116291d0eb3f635" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 46183 } } peers { permanent_uuid: "982377b0de554faf89f0dbff3b55a8d0" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 38615 } }
I20260806 22:46:55.820701 9471 leader_election.cc:288] T d3b81a69eadd4ec294fad3662707cf7c P 4d6b767d87784ec3a9f8ebb281ae463c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3ae77f560314bb69116291d0eb3f635 (127.6.40.66:46183), 982377b0de554faf89f0dbff3b55a8d0 (127.6.40.67:38615)
I20260806 22:46:55.821061 9125 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3b81a69eadd4ec294fad3662707cf7c" candidate_uuid: "4d6b767d87784ec3a9f8ebb281ae463c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3ae77f560314bb69116291d0eb3f635" is_pre_election: true
I20260806 22:46:55.821168 9125 raft_consensus.cc:2454] T d3b81a69eadd4ec294fad3662707cf7c P a3ae77f560314bb69116291d0eb3f635 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d6b767d87784ec3a9f8ebb281ae463c in term 0.
I20260806 22:46:55.821348 8838 leader_election.cc:302] T d3b81a69eadd4ec294fad3662707cf7c P 4d6b767d87784ec3a9f8ebb281ae463c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d6b767d87784ec3a9f8ebb281ae463c, a3ae77f560314bb69116291d0eb3f635; no voters:
I20260806 22:46:55.821437 9471 raft_consensus.cc:2781] T d3b81a69eadd4ec294fad3662707cf7c P 4d6b767d87784ec3a9f8ebb281ae463c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:46:55.821465 9471 raft_consensus.cc:482] T d3b81a69eadd4ec294fad3662707cf7c P 4d6b767d87784ec3a9f8ebb281ae463c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:46:55.821485 9471 raft_consensus.cc:3037] T d3b81a69eadd4ec294fad3662707cf7c P 4d6b767d87784ec3a9f8ebb281ae463c [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:46:55.821507 9320 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3b81a69eadd4ec294fad3662707cf7c" candidate_uuid: "4d6b767d87784ec3a9f8ebb281ae463c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "982377b0de554faf89f0dbff3b55a8d0" is_pre_election: true
I20260806 22:46:55.821642 9320 raft_consensus.cc:2454] T d3b81a69eadd4ec294fad3662707cf7c P 982377b0de554faf89f0dbff3b55a8d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d6b767d87784ec3a9f8ebb281ae463c in term 0.
I20260806 22:46:55.823050 9471 raft_consensus.cc:504] T d3b81a69eadd4ec294fad3662707cf7c P 4d6b767d87784ec3a9f8ebb281ae463c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d6b767d87784ec3a9f8ebb281ae463c" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 33289 } } peers { permanent_uuid: "a3ae77f560314bb69116291d0eb3f635" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 46183 } } peers { permanent_uuid: "982377b0de554faf89f0dbff3b55a8d0" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 38615 } }
I20260806 22:46:55.823195 9471 leader_election.cc:288] T d3b81a69eadd4ec294fad3662707cf7c P 4d6b767d87784ec3a9f8ebb281ae463c [CANDIDATE]: Term 1 election: Requested vote from peers a3ae77f560314bb69116291d0eb3f635 (127.6.40.66:46183), 982377b0de554faf89f0dbff3b55a8d0 (127.6.40.67:38615)
I20260806 22:46:55.823493 9320 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3b81a69eadd4ec294fad3662707cf7c" candidate_uuid: "4d6b767d87784ec3a9f8ebb281ae463c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "982377b0de554faf89f0dbff3b55a8d0"
I20260806 22:46:55.823573 9320 raft_consensus.cc:3037] T d3b81a69eadd4ec294fad3662707cf7c P 982377b0de554faf89f0dbff3b55a8d0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:46:55.823666 9125 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3b81a69eadd4ec294fad3662707cf7c" candidate_uuid: "4d6b767d87784ec3a9f8ebb281ae463c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3ae77f560314bb69116291d0eb3f635"
I20260806 22:46:55.823757 9125 raft_consensus.cc:3037] T d3b81a69eadd4ec294fad3662707cf7c P a3ae77f560314bb69116291d0eb3f635 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:46:55.824357 9320 raft_consensus.cc:2454] T d3b81a69eadd4ec294fad3662707cf7c P 982377b0de554faf89f0dbff3b55a8d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d6b767d87784ec3a9f8ebb281ae463c in term 1.
I20260806 22:46:55.824529 8840 leader_election.cc:302] T d3b81a69eadd4ec294fad3662707cf7c P 4d6b767d87784ec3a9f8ebb281ae463c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d6b767d87784ec3a9f8ebb281ae463c, 982377b0de554faf89f0dbff3b55a8d0; no voters:
I20260806 22:46:55.824688 9471 raft_consensus.cc:2781] T d3b81a69eadd4ec294fad3662707cf7c P 4d6b767d87784ec3a9f8ebb281ae463c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:46:55.824728 9471 raft_consensus.cc:686] T d3b81a69eadd4ec294fad3662707cf7c P 4d6b767d87784ec3a9f8ebb281ae463c [term 1 LEADER]: Becoming Leader. State: Replica: 4d6b767d87784ec3a9f8ebb281ae463c, State: Running, Role: LEADER
I20260806 22:46:55.824790 9471 consensus_queue.cc:237] T d3b81a69eadd4ec294fad3662707cf7c P 4d6b767d87784ec3a9f8ebb281ae463c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d6b767d87784ec3a9f8ebb281ae463c" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 33289 } } peers { permanent_uuid: "a3ae77f560314bb69116291d0eb3f635" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 46183 } } peers { permanent_uuid: "982377b0de554faf89f0dbff3b55a8d0" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 38615 } }
I20260806 22:46:55.825088 9125 raft_consensus.cc:2454] T d3b81a69eadd4ec294fad3662707cf7c P a3ae77f560314bb69116291d0eb3f635 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d6b767d87784ec3a9f8ebb281ae463c in term 1.
I20260806 22:46:55.825460 8715 catalog_manager.cc:5495] T d3b81a69eadd4ec294fad3662707cf7c P 4d6b767d87784ec3a9f8ebb281ae463c reported cstate change: term changed from 0 to 1, leader changed from <none> to 4d6b767d87784ec3a9f8ebb281ae463c (127.6.40.65). New cstate: current_term: 1 leader_uuid: "4d6b767d87784ec3a9f8ebb281ae463c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d6b767d87784ec3a9f8ebb281ae463c" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 33289 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a3ae77f560314bb69116291d0eb3f635" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 46183 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "982377b0de554faf89f0dbff3b55a8d0" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 38615 } health_report { overall_health: UNKNOWN } } }
I20260806 22:46:55.849330 9412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:55.846240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33812 (local address 127.6.40.65:33289)
0806 22:46:55.846317 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:55.846319 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:55.846330 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:46:55.846354 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:55.846355 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:55.846366 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:55.846432 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:46:55.846644 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:55.847372 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:55.848482 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:55.848780 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:55.848878 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:55.849049 (+ 171us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:46:55.849086 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:46:55.849148 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:55.849225 (+ 77us) server_negotiation.cc:299] Negotiation successful
0806 22:46:55.849247 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 22:46:55.855343 9412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:55.852279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.65:36627 (local address 127.6.40.65:33289)
0806 22:46:55.852316 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:55.852318 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:55.852327 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:46:55.852377 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:55.852377 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:55.852387 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:55.852442 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:46:55.852605 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:55.853319 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:55.854531 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:55.854802 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:55.854880 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:55.855050 (+ 170us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:46:55.855082 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:46:55.855142 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:55.855219 (+ 77us) server_negotiation.cc:299] Negotiation successful
0806 22:46:55.855239 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 22:46:55.855365 9469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:55.852213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.65:33289 (local address 127.6.40.65:36627)
0806 22:46:55.852268 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:55.852279 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:55.852322 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:55.852455 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:55.852456 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:55.852458 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 22:46:55.852525 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:55.852527 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:55.853422 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:55.853423 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:55.854412 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:55.854416 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:55.854899 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:55.854900 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:55.854955 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:55.854961 (+ 6us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 22:46:55.855162 (+ 201us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 22:46:55.855164 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 22:46:55.855216 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 22:46:55.855264 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260806 22:46:55.858275 9320 raft_consensus.cc:1261] T d3b81a69eadd4ec294fad3662707cf7c P 982377b0de554faf89f0dbff3b55a8d0 [term 1 FOLLOWER]: Refusing update from remote peer 4d6b767d87784ec3a9f8ebb281ae463c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:46:55.858489 9444 consensus_queue.cc:1035] T d3b81a69eadd4ec294fad3662707cf7c P 4d6b767d87784ec3a9f8ebb281ae463c [LEADER]: Connected to new peer: Peer: permanent_uuid: "982377b0de554faf89f0dbff3b55a8d0" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 38615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:46:55.859956 9455 mvcc.cc:205] Tried to move back new op lower bound from 7315687079353315328 to 7315687079218794496. Current Snapshot: MvccSnapshot[applied={T|T < 7315687079353315328}]
I20260806 22:46:55.861258 9125 raft_consensus.cc:1261] T d3b81a69eadd4ec294fad3662707cf7c P a3ae77f560314bb69116291d0eb3f635 [term 1 FOLLOWER]: Refusing update from remote peer 4d6b767d87784ec3a9f8ebb281ae463c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:46:55.861481 9471 consensus_queue.cc:1035] T d3b81a69eadd4ec294fad3662707cf7c P 4d6b767d87784ec3a9f8ebb281ae463c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3ae77f560314bb69116291d0eb3f635" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 46183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 22:46:55.865702 9451 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260806 22:46:57.391204 8715 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40564:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:46:57.391366 8715 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40564:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:46:57.392412 8715 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9d0e72021aa443289c0bb45de789d382: Sending DeleteTablet for 3 replicas of tablet d3b81a69eadd4ec294fad3662707cf7c
I20260806 22:46:57.392733 8902 tablet_service.cc:1514] Processing DeleteTablet for tablet d3b81a69eadd4ec294fad3662707cf7c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:46:57 UTC) from {username='kudu', principal='kudu/127.6.40.126@KRBTEST.COM'} at 127.0.0.1:33792
I20260806 22:46:57.392776 9293 tablet_service.cc:1514] Processing DeleteTablet for tablet d3b81a69eadd4ec294fad3662707cf7c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:46:57 UTC) from {username='kudu', principal='kudu/127.6.40.126@KRBTEST.COM'} at 127.0.0.1:57548
I20260806 22:46:57.392779 9098 tablet_service.cc:1514] Processing DeleteTablet for tablet d3b81a69eadd4ec294fad3662707cf7c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:46:57 UTC) from {username='kudu', principal='kudu/127.6.40.126@KRBTEST.COM'} at 127.0.0.1:51600
I20260806 22:46:57.393083 9517 tablet_replica.cc:330] T d3b81a69eadd4ec294fad3662707cf7c P 4d6b767d87784ec3a9f8ebb281ae463c: stopping tablet replica
I20260806 22:46:57.393172 9517 raft_consensus.cc:2229] T d3b81a69eadd4ec294fad3662707cf7c P 4d6b767d87784ec3a9f8ebb281ae463c [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:46:57.393265 9517 raft_consensus.cc:2258] T d3b81a69eadd4ec294fad3662707cf7c P 4d6b767d87784ec3a9f8ebb281ae463c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:46:57.393414 9518 tablet_replica.cc:330] T d3b81a69eadd4ec294fad3662707cf7c P a3ae77f560314bb69116291d0eb3f635: stopping tablet replica
I20260806 22:46:57.393492 9518 raft_consensus.cc:2229] T d3b81a69eadd4ec294fad3662707cf7c P a3ae77f560314bb69116291d0eb3f635 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:46:57.393543 9518 raft_consensus.cc:2258] T d3b81a69eadd4ec294fad3662707cf7c P a3ae77f560314bb69116291d0eb3f635 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:46:57.393811 9517 ts_tablet_manager.cc:1905] T d3b81a69eadd4ec294fad3662707cf7c P 4d6b767d87784ec3a9f8ebb281ae463c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:46:57.395567 9519 tablet_replica.cc:330] T d3b81a69eadd4ec294fad3662707cf7c P 982377b0de554faf89f0dbff3b55a8d0: stopping tablet replica
I20260806 22:46:57.395661 9519 raft_consensus.cc:2229] T d3b81a69eadd4ec294fad3662707cf7c P 982377b0de554faf89f0dbff3b55a8d0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:46:57.395660 9517 ts_tablet_manager.cc:1918] T d3b81a69eadd4ec294fad3662707cf7c P 4d6b767d87784ec3a9f8ebb281ae463c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 22:46:57.395722 9517 log.cc:1196] T d3b81a69eadd4ec294fad3662707cf7c P 4d6b767d87784ec3a9f8ebb281ae463c: Deleting WAL directory at /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/minicluster-data/ts-0/wal/wals/d3b81a69eadd4ec294fad3662707cf7c
I20260806 22:46:57.395722 9519 raft_consensus.cc:2258] T d3b81a69eadd4ec294fad3662707cf7c P 982377b0de554faf89f0dbff3b55a8d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:46:57.396001 9518 ts_tablet_manager.cc:1905] T d3b81a69eadd4ec294fad3662707cf7c P a3ae77f560314bb69116291d0eb3f635: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:46:57.396265 9519 ts_tablet_manager.cc:1905] T d3b81a69eadd4ec294fad3662707cf7c P 982377b0de554faf89f0dbff3b55a8d0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:46:57.396313 9517 ts_tablet_manager.cc:1939] T d3b81a69eadd4ec294fad3662707cf7c P 4d6b767d87784ec3a9f8ebb281ae463c: Deleting consensus metadata
I20260806 22:46:57.396761 8698 catalog_manager.cc:4841] TS 4d6b767d87784ec3a9f8ebb281ae463c (127.6.40.65:33289): tablet d3b81a69eadd4ec294fad3662707cf7c (table test-table [id=962c13b0eb904d6cab92a0c8c06c62f8]) successfully deleted
I20260806 22:46:57.399369 9519 ts_tablet_manager.cc:1918] T d3b81a69eadd4ec294fad3662707cf7c P 982377b0de554faf89f0dbff3b55a8d0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 22:46:57.401261 9518 ts_tablet_manager.cc:1918] T d3b81a69eadd4ec294fad3662707cf7c P a3ae77f560314bb69116291d0eb3f635: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 22:46:57.401366 9518 log.cc:1196] T d3b81a69eadd4ec294fad3662707cf7c P a3ae77f560314bb69116291d0eb3f635: Deleting WAL directory at /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/minicluster-data/ts-1/wal/wals/d3b81a69eadd4ec294fad3662707cf7c
I20260806 22:46:57.401659 9518 ts_tablet_manager.cc:1939] T d3b81a69eadd4ec294fad3662707cf7c P a3ae77f560314bb69116291d0eb3f635: Deleting consensus metadata
I20260806 22:46:57.402019 9519 log.cc:1196] T d3b81a69eadd4ec294fad3662707cf7c P 982377b0de554faf89f0dbff3b55a8d0: Deleting WAL directory at /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/minicluster-data/ts-2/wal/wals/d3b81a69eadd4ec294fad3662707cf7c
I20260806 22:46:57.402089 8696 catalog_manager.cc:4841] TS a3ae77f560314bb69116291d0eb3f635 (127.6.40.66:46183): tablet d3b81a69eadd4ec294fad3662707cf7c (table test-table [id=962c13b0eb904d6cab92a0c8c06c62f8]) successfully deleted
I20260806 22:46:57.402331 9519 ts_tablet_manager.cc:1939] T d3b81a69eadd4ec294fad3662707cf7c P 982377b0de554faf89f0dbff3b55a8d0: Deleting consensus metadata
I20260806 22:46:57.402735 8697 catalog_manager.cc:4841] TS 982377b0de554faf89f0dbff3b55a8d0 (127.6.40.67:38615): tablet d3b81a69eadd4ec294fad3662707cf7c (table test-table [id=962c13b0eb904d6cab92a0c8c06c62f8]) successfully deleted
Aug 06 22:46:57 dist-test-slave-zh2d krb5kdc[8654](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056417, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 22:46:57 dist-test-slave-zh2d krb5kdc[8654](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056417, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
I20260806 22:46:57.426376 9529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:57.413445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40584 (local address 127.6.40.126:39721)
0806 22:46:57.413611 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:57.413613 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:57.413641 (+ 28us) server_negotiation.cc:407] Connection header received
0806 22:46:57.413722 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:57.413723 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:57.413737 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:57.413804 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:57.414425 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:57.414948 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:57.415687 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:57.415893 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:57.415951 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:57.423482 (+ 7531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:57.423484 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:57.423485 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:46:57.423501 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:57.425091 (+ 1590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:57.425466 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:57.425468 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:57.425469 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:57.425546 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:57.425768 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:57.425769 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:57.425769 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:57.425894 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:57.426009 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:57.426168 (+ 159us) server_negotiation.cc:299] Negotiation successful
0806 22:46:57.426260 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":99,"threads_started":1}
I20260806 22:46:57.428381 8715 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40584:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:46:57.428547 8715 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:46:57.430332 9293 tablet_service.cc:1467] Processing CreateTablet for tablet 08b1b57d10224de887380b2d78932d1c (DEFAULT_TABLE table=test-table [id=3335bc2f391043788441d0ef84c7002f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:46:57.430385 9098 tablet_service.cc:1467] Processing CreateTablet for tablet 08b1b57d10224de887380b2d78932d1c (DEFAULT_TABLE table=test-table [id=3335bc2f391043788441d0ef84c7002f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:46:57.430521 8902 tablet_service.cc:1467] Processing CreateTablet for tablet 08b1b57d10224de887380b2d78932d1c (DEFAULT_TABLE table=test-table [id=3335bc2f391043788441d0ef84c7002f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:46:57.430570 9293 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08b1b57d10224de887380b2d78932d1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:46:57.430601 9098 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08b1b57d10224de887380b2d78932d1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:46:57.430681 8902 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08b1b57d10224de887380b2d78932d1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:46:57.432037 9533 tablet_bootstrap.cc:492] T 08b1b57d10224de887380b2d78932d1c P 982377b0de554faf89f0dbff3b55a8d0: Bootstrap starting.
I20260806 22:46:57.432366 9533 tablet_bootstrap.cc:654] T 08b1b57d10224de887380b2d78932d1c P 982377b0de554faf89f0dbff3b55a8d0: Neither blocks nor log segments found. Creating new log.
I20260806 22:46:57.432524 9534 tablet_bootstrap.cc:492] T 08b1b57d10224de887380b2d78932d1c P 4d6b767d87784ec3a9f8ebb281ae463c: Bootstrap starting.
I20260806 22:46:57.432644 9536 tablet_bootstrap.cc:492] T 08b1b57d10224de887380b2d78932d1c P a3ae77f560314bb69116291d0eb3f635: Bootstrap starting.
I20260806 22:46:57.432956 9534 tablet_bootstrap.cc:654] T 08b1b57d10224de887380b2d78932d1c P 4d6b767d87784ec3a9f8ebb281ae463c: Neither blocks nor log segments found. Creating new log.
I20260806 22:46:57.433322 9533 tablet_bootstrap.cc:492] T 08b1b57d10224de887380b2d78932d1c P 982377b0de554faf89f0dbff3b55a8d0: No bootstrap required, opened a new log
I20260806 22:46:57.433528 9533 ts_tablet_manager.cc:1397] T 08b1b57d10224de887380b2d78932d1c P 982377b0de554faf89f0dbff3b55a8d0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:46:57.433668 9536 tablet_bootstrap.cc:654] T 08b1b57d10224de887380b2d78932d1c P a3ae77f560314bb69116291d0eb3f635: Neither blocks nor log segments found. Creating new log.
I20260806 22:46:57.433754 9534 tablet_bootstrap.cc:492] T 08b1b57d10224de887380b2d78932d1c P 4d6b767d87784ec3a9f8ebb281ae463c: No bootstrap required, opened a new log
I20260806 22:46:57.433840 9534 ts_tablet_manager.cc:1397] T 08b1b57d10224de887380b2d78932d1c P 4d6b767d87784ec3a9f8ebb281ae463c: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260806 22:46:57.433835 9533 raft_consensus.cc:348] T 08b1b57d10224de887380b2d78932d1c P 982377b0de554faf89f0dbff3b55a8d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d6b767d87784ec3a9f8ebb281ae463c" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 33289 } } peers { permanent_uuid: "a3ae77f560314bb69116291d0eb3f635" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 46183 } } peers { permanent_uuid: "982377b0de554faf89f0dbff3b55a8d0" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 38615 } }
I20260806 22:46:57.433934 9533 raft_consensus.cc:374] T 08b1b57d10224de887380b2d78932d1c P 982377b0de554faf89f0dbff3b55a8d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:46:57.433990 9533 raft_consensus.cc:729] T 08b1b57d10224de887380b2d78932d1c P 982377b0de554faf89f0dbff3b55a8d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 982377b0de554faf89f0dbff3b55a8d0, State: Initialized, Role: FOLLOWER
I20260806 22:46:57.434001 9534 raft_consensus.cc:348] T 08b1b57d10224de887380b2d78932d1c P 4d6b767d87784ec3a9f8ebb281ae463c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d6b767d87784ec3a9f8ebb281ae463c" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 33289 } } peers { permanent_uuid: "a3ae77f560314bb69116291d0eb3f635" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 46183 } } peers { permanent_uuid: "982377b0de554faf89f0dbff3b55a8d0" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 38615 } }
I20260806 22:46:57.434104 9534 raft_consensus.cc:374] T 08b1b57d10224de887380b2d78932d1c P 4d6b767d87784ec3a9f8ebb281ae463c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:46:57.434139 9534 raft_consensus.cc:729] T 08b1b57d10224de887380b2d78932d1c P 4d6b767d87784ec3a9f8ebb281ae463c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d6b767d87784ec3a9f8ebb281ae463c, State: Initialized, Role: FOLLOWER
I20260806 22:46:57.434105 9533 consensus_queue.cc:260] T 08b1b57d10224de887380b2d78932d1c P 982377b0de554faf89f0dbff3b55a8d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d6b767d87784ec3a9f8ebb281ae463c" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 33289 } } peers { permanent_uuid: "a3ae77f560314bb69116291d0eb3f635" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 46183 } } peers { permanent_uuid: "982377b0de554faf89f0dbff3b55a8d0" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 38615 } }
I20260806 22:46:57.434254 9534 consensus_queue.cc:260] T 08b1b57d10224de887380b2d78932d1c P 4d6b767d87784ec3a9f8ebb281ae463c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d6b767d87784ec3a9f8ebb281ae463c" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 33289 } } peers { permanent_uuid: "a3ae77f560314bb69116291d0eb3f635" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 46183 } } peers { permanent_uuid: "982377b0de554faf89f0dbff3b55a8d0" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 38615 } }
I20260806 22:46:57.434326 9533 ts_tablet_manager.cc:1428] T 08b1b57d10224de887380b2d78932d1c P 982377b0de554faf89f0dbff3b55a8d0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 22:46:57.434476 9534 ts_tablet_manager.cc:1428] T 08b1b57d10224de887380b2d78932d1c P 4d6b767d87784ec3a9f8ebb281ae463c: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260806 22:46:57.434607 9536 tablet_bootstrap.cc:492] T 08b1b57d10224de887380b2d78932d1c P a3ae77f560314bb69116291d0eb3f635: No bootstrap required, opened a new log
I20260806 22:46:57.434679 9536 ts_tablet_manager.cc:1397] T 08b1b57d10224de887380b2d78932d1c P a3ae77f560314bb69116291d0eb3f635: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:46:57.434816 9536 raft_consensus.cc:348] T 08b1b57d10224de887380b2d78932d1c P a3ae77f560314bb69116291d0eb3f635 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d6b767d87784ec3a9f8ebb281ae463c" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 33289 } } peers { permanent_uuid: "a3ae77f560314bb69116291d0eb3f635" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 46183 } } peers { permanent_uuid: "982377b0de554faf89f0dbff3b55a8d0" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 38615 } }
I20260806 22:46:57.434948 9536 raft_consensus.cc:374] T 08b1b57d10224de887380b2d78932d1c P a3ae77f560314bb69116291d0eb3f635 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:46:57.435006 9536 raft_consensus.cc:729] T 08b1b57d10224de887380b2d78932d1c P a3ae77f560314bb69116291d0eb3f635 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3ae77f560314bb69116291d0eb3f635, State: Initialized, Role: FOLLOWER
I20260806 22:46:57.435124 9536 consensus_queue.cc:260] T 08b1b57d10224de887380b2d78932d1c P a3ae77f560314bb69116291d0eb3f635 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d6b767d87784ec3a9f8ebb281ae463c" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 33289 } } peers { permanent_uuid: "a3ae77f560314bb69116291d0eb3f635" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 46183 } } peers { permanent_uuid: "982377b0de554faf89f0dbff3b55a8d0" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 38615 } }
I20260806 22:46:57.435443 9536 ts_tablet_manager.cc:1428] T 08b1b57d10224de887380b2d78932d1c P a3ae77f560314bb69116291d0eb3f635: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 22:46:57.606298 9542 raft_consensus.cc:482] T 08b1b57d10224de887380b2d78932d1c P a3ae77f560314bb69116291d0eb3f635 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:46:57.606429 9542 raft_consensus.cc:504] T 08b1b57d10224de887380b2d78932d1c P a3ae77f560314bb69116291d0eb3f635 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d6b767d87784ec3a9f8ebb281ae463c" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 33289 } } peers { permanent_uuid: "a3ae77f560314bb69116291d0eb3f635" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 46183 } } peers { permanent_uuid: "982377b0de554faf89f0dbff3b55a8d0" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 38615 } }
I20260806 22:46:57.606906 9542 leader_election.cc:288] T 08b1b57d10224de887380b2d78932d1c P a3ae77f560314bb69116291d0eb3f635 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4d6b767d87784ec3a9f8ebb281ae463c (127.6.40.65:33289), 982377b0de554faf89f0dbff3b55a8d0 (127.6.40.67:38615)
I20260806 22:46:57.610553 9545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:57.607114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.67:38615 (local address 127.6.40.66:32999)
0806 22:46:57.607400 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:57.607416 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:57.607497 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:57.607717 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:57.607718 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:57.607720 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:46:57.607813 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:57.607815 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:57.608555 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:57.608556 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:57.609564 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:57.609567 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:57.610235 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:57.610236 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:57.610310 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:57.610314 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:46:57.610399 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 22:46:57.610449 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"mutex_wait_us":5,"thread_start_us":179,"threads_started":1}
I20260806 22:46:57.610540 9548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:57.607243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.66:32999 (local address 127.6.40.67:38615)
0806 22:46:57.607428 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:57.607430 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:57.607475 (+ 45us) server_negotiation.cc:407] Connection header received
0806 22:46:57.607602 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:57.607603 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:57.607624 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:57.607714 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:46:57.607906 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:57.608460 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:57.609676 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:57.610110 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:57.610205 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:57.610221 (+ 16us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:57.610406 (+ 185us) server_negotiation.cc:299] Negotiation successful
0806 22:46:57.610454 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":99,"threads_started":1}
I20260806 22:46:57.610873 9320 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08b1b57d10224de887380b2d78932d1c" candidate_uuid: "a3ae77f560314bb69116291d0eb3f635" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "982377b0de554faf89f0dbff3b55a8d0" is_pre_election: true
I20260806 22:46:57.611006 9320 raft_consensus.cc:2454] T 08b1b57d10224de887380b2d78932d1c P 982377b0de554faf89f0dbff3b55a8d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3ae77f560314bb69116291d0eb3f635 in term 0.
I20260806 22:46:57.611126 9544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:57.607122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.65:33289 (local address 127.6.40.66:51837)
0806 22:46:57.607504 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:57.607513 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:57.607596 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:57.607799 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:57.607800 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:57.607802 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:46:57.608228 (+ 426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:57.608230 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:57.609151 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:57.609153 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:57.610067 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:57.610070 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:57.610856 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:57.610857 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:57.610940 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:57.610944 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:46:57.610999 (+ 55us) client_negotiation.cc:240] Negotiation successful
0806 22:46:57.611042 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":167,"threads_started":1}
I20260806 22:46:57.611202 9546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:57.607191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.66:51837 (local address 127.6.40.65:33289)
0806 22:46:57.607452 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:57.607454 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:57.607580 (+ 126us) server_negotiation.cc:407] Connection header received
0806 22:46:57.607656 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:57.607657 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:57.607671 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:57.607795 (+ 124us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:46:57.608325 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:57.608894 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:57.610279 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:57.610743 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:57.610824 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:57.610838 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:57.611063 (+ 225us) server_negotiation.cc:299] Negotiation successful
0806 22:46:57.611123 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":131,"threads_started":1}
I20260806 22:46:57.611224 9027 leader_election.cc:302] T 08b1b57d10224de887380b2d78932d1c P a3ae77f560314bb69116291d0eb3f635 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 982377b0de554faf89f0dbff3b55a8d0, a3ae77f560314bb69116291d0eb3f635; no voters:
I20260806 22:46:57.611377 9542 raft_consensus.cc:2781] T 08b1b57d10224de887380b2d78932d1c P a3ae77f560314bb69116291d0eb3f635 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:46:57.611407 9542 raft_consensus.cc:482] T 08b1b57d10224de887380b2d78932d1c P a3ae77f560314bb69116291d0eb3f635 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:46:57.611425 9542 raft_consensus.cc:3037] T 08b1b57d10224de887380b2d78932d1c P a3ae77f560314bb69116291d0eb3f635 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:46:57.611374 8933 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08b1b57d10224de887380b2d78932d1c" candidate_uuid: "a3ae77f560314bb69116291d0eb3f635" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d6b767d87784ec3a9f8ebb281ae463c" is_pre_election: true
I20260806 22:46:57.611465 8933 raft_consensus.cc:2454] T 08b1b57d10224de887380b2d78932d1c P 4d6b767d87784ec3a9f8ebb281ae463c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3ae77f560314bb69116291d0eb3f635 in term 0.
I20260806 22:46:57.612404 9542 raft_consensus.cc:504] T 08b1b57d10224de887380b2d78932d1c P a3ae77f560314bb69116291d0eb3f635 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d6b767d87784ec3a9f8ebb281ae463c" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 33289 } } peers { permanent_uuid: "a3ae77f560314bb69116291d0eb3f635" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 46183 } } peers { permanent_uuid: "982377b0de554faf89f0dbff3b55a8d0" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 38615 } }
I20260806 22:46:57.612591 9542 leader_election.cc:288] T 08b1b57d10224de887380b2d78932d1c P a3ae77f560314bb69116291d0eb3f635 [CANDIDATE]: Term 1 election: Requested vote from peers 4d6b767d87784ec3a9f8ebb281ae463c (127.6.40.65:33289), 982377b0de554faf89f0dbff3b55a8d0 (127.6.40.67:38615)
I20260806 22:46:57.612735 8933 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08b1b57d10224de887380b2d78932d1c" candidate_uuid: "a3ae77f560314bb69116291d0eb3f635" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d6b767d87784ec3a9f8ebb281ae463c"
I20260806 22:46:57.612877 8933 raft_consensus.cc:3037] T 08b1b57d10224de887380b2d78932d1c P 4d6b767d87784ec3a9f8ebb281ae463c [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:46:57.612874 9320 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08b1b57d10224de887380b2d78932d1c" candidate_uuid: "a3ae77f560314bb69116291d0eb3f635" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "982377b0de554faf89f0dbff3b55a8d0"
I20260806 22:46:57.612988 9320 raft_consensus.cc:3037] T 08b1b57d10224de887380b2d78932d1c P 982377b0de554faf89f0dbff3b55a8d0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:46:57.613909 8933 raft_consensus.cc:2454] T 08b1b57d10224de887380b2d78932d1c P 4d6b767d87784ec3a9f8ebb281ae463c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3ae77f560314bb69116291d0eb3f635 in term 1.
I20260806 22:46:57.613909 9320 raft_consensus.cc:2454] T 08b1b57d10224de887380b2d78932d1c P 982377b0de554faf89f0dbff3b55a8d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3ae77f560314bb69116291d0eb3f635 in term 1.
I20260806 22:46:57.614125 9027 leader_election.cc:302] T 08b1b57d10224de887380b2d78932d1c P a3ae77f560314bb69116291d0eb3f635 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 982377b0de554faf89f0dbff3b55a8d0, a3ae77f560314bb69116291d0eb3f635; no voters:
I20260806 22:46:57.614274 9542 raft_consensus.cc:2781] T 08b1b57d10224de887380b2d78932d1c P a3ae77f560314bb69116291d0eb3f635 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:46:57.614449 9542 raft_consensus.cc:686] T 08b1b57d10224de887380b2d78932d1c P a3ae77f560314bb69116291d0eb3f635 [term 1 LEADER]: Becoming Leader. State: Replica: a3ae77f560314bb69116291d0eb3f635, State: Running, Role: LEADER
I20260806 22:46:57.614559 9542 consensus_queue.cc:237] T 08b1b57d10224de887380b2d78932d1c P a3ae77f560314bb69116291d0eb3f635 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d6b767d87784ec3a9f8ebb281ae463c" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 33289 } } peers { permanent_uuid: "a3ae77f560314bb69116291d0eb3f635" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 46183 } } peers { permanent_uuid: "982377b0de554faf89f0dbff3b55a8d0" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 38615 } }
I20260806 22:46:57.615326 8716 catalog_manager.cc:5495] T 08b1b57d10224de887380b2d78932d1c P a3ae77f560314bb69116291d0eb3f635 reported cstate change: term changed from 0 to 1, leader changed from <none> to a3ae77f560314bb69116291d0eb3f635 (127.6.40.66). New cstate: current_term: 1 leader_uuid: "a3ae77f560314bb69116291d0eb3f635" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d6b767d87784ec3a9f8ebb281ae463c" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 33289 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3ae77f560314bb69116291d0eb3f635" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 46183 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "982377b0de554faf89f0dbff3b55a8d0" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 38615 } health_report { overall_health: UNKNOWN } } }
I20260806 22:46:57.667711 9552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:57.663191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51614 (local address 127.6.40.66:46183)
0806 22:46:57.664936 (+ 1745us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:57.664939 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:57.664948 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:46:57.664990 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:57.664991 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:57.665012 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:57.665089 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:46:57.665251 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:57.665794 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:57.666602 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:57.666828 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:57.666945 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:57.667180 (+ 235us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:46:57.667354 (+ 174us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:46:57.667442 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:57.667524 (+ 82us) server_negotiation.cc:299] Negotiation successful
0806 22:46:57.667553 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1697,"thread_start_us":1566,"threads_started":1}
I20260806 22:46:57.674979 9546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:57.671216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.66:43285 (local address 127.6.40.65:33289)
0806 22:46:57.671288 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:57.671290 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:57.671395 (+ 105us) server_negotiation.cc:407] Connection header received
0806 22:46:57.671504 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:57.671505 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:57.671520 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:57.671571 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:46:57.671790 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:57.672647 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:57.674083 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:57.674409 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:57.674483 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:57.674694 (+ 211us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:46:57.674733 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:46:57.674791 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:57.674868 (+ 77us) server_negotiation.cc:299] Negotiation successful
0806 22:46:57.674892 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 22:46:57.675033 9555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:57.671050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.65:33289 (local address 127.6.40.66:43285)
0806 22:46:57.671333 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:57.671344 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:57.671405 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:57.671583 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:57.671584 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:57.671586 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 22:46:57.671680 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:57.671682 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:57.672756 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:57.672758 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:57.673983 (+ 1225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:57.673987 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:57.674500 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:57.674501 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:57.674581 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:57.674597 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 22:46:57.674807 (+ 210us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 22:46:57.674810 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:46:57.674860 (+ 50us) client_negotiation.cc:240] Negotiation successful
0806 22:46:57.674912 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":124,"threads_started":1}
I20260806 22:46:57.677873 9320 raft_consensus.cc:1261] T 08b1b57d10224de887380b2d78932d1c P 982377b0de554faf89f0dbff3b55a8d0 [term 1 FOLLOWER]: Refusing update from remote peer a3ae77f560314bb69116291d0eb3f635: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:46:57.677956 8933 raft_consensus.cc:1261] T 08b1b57d10224de887380b2d78932d1c P 4d6b767d87784ec3a9f8ebb281ae463c [term 1 FOLLOWER]: Refusing update from remote peer a3ae77f560314bb69116291d0eb3f635: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:46:57.678256 9542 consensus_queue.cc:1035] T 08b1b57d10224de887380b2d78932d1c P a3ae77f560314bb69116291d0eb3f635 [LEADER]: Connected to new peer: Peer: permanent_uuid: "982377b0de554faf89f0dbff3b55a8d0" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 38615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:46:57.678440 9549 consensus_queue.cc:1035] T 08b1b57d10224de887380b2d78932d1c P a3ae77f560314bb69116291d0eb3f635 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d6b767d87784ec3a9f8ebb281ae463c" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 33289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:46:57.686772 9552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:57.684012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.65:43049 (local address 127.6.40.66:46183)
0806 22:46:57.684132 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:57.684135 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:57.684438 (+ 303us) server_negotiation.cc:407] Connection header received
0806 22:46:57.684494 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:57.684495 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:57.684507 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:57.684563 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:46:57.684693 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:57.685242 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:57.686047 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:57.686259 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:57.686337 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:57.686500 (+ 163us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:46:57.686535 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:46:57.686612 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:57.686675 (+ 63us) server_negotiation.cc:299] Negotiation successful
0806 22:46:57.686696 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260806 22:46:57.686807 9561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:57.683973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.66:46183 (local address 127.6.40.65:43049)
0806 22:46:57.684308 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:57.684319 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:57.684435 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:57.684567 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:57.684568 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:57.684570 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 22:46:57.684629 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:57.684631 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:57.685328 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:57.685329 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:57.685952 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:57.685954 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:57.686344 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:57.686346 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:57.686422 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:57.686426 (+ 4us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 22:46:57.686616 (+ 190us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 22:46:57.686619 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:46:57.686670 (+ 51us) client_negotiation.cc:240] Negotiation successful
0806 22:46:57.686723 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":105,"threads_started":1}
I20260806 22:46:57.694458 8716 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40584:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:46:57.694576 8716 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40584:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:46:57.695262 8716 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9d0e72021aa443289c0bb45de789d382: Sending DeleteTablet for 3 replicas of tablet 08b1b57d10224de887380b2d78932d1c
I20260806 22:46:57.695545 8902 tablet_service.cc:1514] Processing DeleteTablet for tablet 08b1b57d10224de887380b2d78932d1c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:46:57 UTC) from {username='kudu', principal='kudu/127.6.40.126@KRBTEST.COM'} at 127.0.0.1:33792
I20260806 22:46:57.695552 9293 tablet_service.cc:1514] Processing DeleteTablet for tablet 08b1b57d10224de887380b2d78932d1c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:46:57 UTC) from {username='kudu', principal='kudu/127.6.40.126@KRBTEST.COM'} at 127.0.0.1:57548
I20260806 22:46:57.695655 9519 tablet_replica.cc:330] T 08b1b57d10224de887380b2d78932d1c P 982377b0de554faf89f0dbff3b55a8d0: stopping tablet replica
I20260806 22:46:57.695665 9517 tablet_replica.cc:330] T 08b1b57d10224de887380b2d78932d1c P 4d6b767d87784ec3a9f8ebb281ae463c: stopping tablet replica
I20260806 22:46:57.695724 9517 raft_consensus.cc:2229] T 08b1b57d10224de887380b2d78932d1c P 4d6b767d87784ec3a9f8ebb281ae463c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:46:57.695720 9519 raft_consensus.cc:2229] T 08b1b57d10224de887380b2d78932d1c P 982377b0de554faf89f0dbff3b55a8d0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:46:57.695761 9517 raft_consensus.cc:2258] T 08b1b57d10224de887380b2d78932d1c P 4d6b767d87784ec3a9f8ebb281ae463c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:46:57.695768 9519 raft_consensus.cc:2258] T 08b1b57d10224de887380b2d78932d1c P 982377b0de554faf89f0dbff3b55a8d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:46:57.695781 9098 tablet_service.cc:1514] Processing DeleteTablet for tablet 08b1b57d10224de887380b2d78932d1c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:46:57 UTC) from {username='kudu', principal='kudu/127.6.40.126@KRBTEST.COM'} at 127.0.0.1:51600
I20260806 22:46:57.695881 9518 tablet_replica.cc:330] T 08b1b57d10224de887380b2d78932d1c P a3ae77f560314bb69116291d0eb3f635: stopping tablet replica
I20260806 22:46:57.695930 9518 raft_consensus.cc:2229] T 08b1b57d10224de887380b2d78932d1c P a3ae77f560314bb69116291d0eb3f635 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:46:57.696018 9518 raft_consensus.cc:2258] T 08b1b57d10224de887380b2d78932d1c P a3ae77f560314bb69116291d0eb3f635 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:46:57.696138 9517 ts_tablet_manager.cc:1905] T 08b1b57d10224de887380b2d78932d1c P 4d6b767d87784ec3a9f8ebb281ae463c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:46:57.696151 9519 ts_tablet_manager.cc:1905] T 08b1b57d10224de887380b2d78932d1c P 982377b0de554faf89f0dbff3b55a8d0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:46:57.696317 9518 ts_tablet_manager.cc:1905] T 08b1b57d10224de887380b2d78932d1c P a3ae77f560314bb69116291d0eb3f635: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:46:57.697939 9519 ts_tablet_manager.cc:1918] T 08b1b57d10224de887380b2d78932d1c P 982377b0de554faf89f0dbff3b55a8d0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:46:57.698002 9519 log.cc:1196] T 08b1b57d10224de887380b2d78932d1c P 982377b0de554faf89f0dbff3b55a8d0: Deleting WAL directory at /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/minicluster-data/ts-2/wal/wals/08b1b57d10224de887380b2d78932d1c
I20260806 22:46:57.698151 9517 ts_tablet_manager.cc:1918] T 08b1b57d10224de887380b2d78932d1c P 4d6b767d87784ec3a9f8ebb281ae463c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:46:57.698208 9517 log.cc:1196] T 08b1b57d10224de887380b2d78932d1c P 4d6b767d87784ec3a9f8ebb281ae463c: Deleting WAL directory at /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/minicluster-data/ts-0/wal/wals/08b1b57d10224de887380b2d78932d1c
I20260806 22:46:57.698235 9519 ts_tablet_manager.cc:1939] T 08b1b57d10224de887380b2d78932d1c P 982377b0de554faf89f0dbff3b55a8d0: Deleting consensus metadata
I20260806 22:46:57.698249 9518 ts_tablet_manager.cc:1918] T 08b1b57d10224de887380b2d78932d1c P a3ae77f560314bb69116291d0eb3f635: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:46:57.698298 9518 log.cc:1196] T 08b1b57d10224de887380b2d78932d1c P a3ae77f560314bb69116291d0eb3f635: Deleting WAL directory at /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/minicluster-data/ts-1/wal/wals/08b1b57d10224de887380b2d78932d1c
I20260806 22:46:57.698473 9517 ts_tablet_manager.cc:1939] T 08b1b57d10224de887380b2d78932d1c P 4d6b767d87784ec3a9f8ebb281ae463c: Deleting consensus metadata
I20260806 22:46:57.698519 9518 ts_tablet_manager.cc:1939] T 08b1b57d10224de887380b2d78932d1c P a3ae77f560314bb69116291d0eb3f635: Deleting consensus metadata
I20260806 22:46:57.698832 8697 catalog_manager.cc:4841] TS 982377b0de554faf89f0dbff3b55a8d0 (127.6.40.67:38615): tablet 08b1b57d10224de887380b2d78932d1c (table test-table [id=3335bc2f391043788441d0ef84c7002f]) successfully deleted
I20260806 22:46:57.698947 8698 catalog_manager.cc:4841] TS 4d6b767d87784ec3a9f8ebb281ae463c (127.6.40.65:33289): tablet 08b1b57d10224de887380b2d78932d1c (table test-table [id=3335bc2f391043788441d0ef84c7002f]) successfully deleted
I20260806 22:46:57.699067 8696 catalog_manager.cc:4841] TS a3ae77f560314bb69116291d0eb3f635 (127.6.40.66:46183): tablet 08b1b57d10224de887380b2d78932d1c (table test-table [id=3335bc2f391043788441d0ef84c7002f]) successfully deleted
I20260806 22:46:57.699205 8716 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40584:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:46:57.699331 8716 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:46:57.700662 8902 tablet_service.cc:1467] Processing CreateTablet for tablet 2740dde8504b47ca9a96707481cce2da (DEFAULT_TABLE table=test-table [id=b8f50e53088047899afb7085a24a89b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:46:57.700706 9293 tablet_service.cc:1467] Processing CreateTablet for tablet 2740dde8504b47ca9a96707481cce2da (DEFAULT_TABLE table=test-table [id=b8f50e53088047899afb7085a24a89b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:46:57.700711 9098 tablet_service.cc:1467] Processing CreateTablet for tablet 2740dde8504b47ca9a96707481cce2da (DEFAULT_TABLE table=test-table [id=b8f50e53088047899afb7085a24a89b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:46:57.700788 8902 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2740dde8504b47ca9a96707481cce2da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:46:57.700814 9098 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2740dde8504b47ca9a96707481cce2da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:46:57.700814 9293 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2740dde8504b47ca9a96707481cce2da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:46:57.702078 9533 tablet_bootstrap.cc:492] T 2740dde8504b47ca9a96707481cce2da P 982377b0de554faf89f0dbff3b55a8d0: Bootstrap starting.
I20260806 22:46:57.702082 9536 tablet_bootstrap.cc:492] T 2740dde8504b47ca9a96707481cce2da P a3ae77f560314bb69116291d0eb3f635: Bootstrap starting.
I20260806 22:46:57.702620 9533 tablet_bootstrap.cc:654] T 2740dde8504b47ca9a96707481cce2da P 982377b0de554faf89f0dbff3b55a8d0: Neither blocks nor log segments found. Creating new log.
I20260806 22:46:57.702965 9534 tablet_bootstrap.cc:492] T 2740dde8504b47ca9a96707481cce2da P 4d6b767d87784ec3a9f8ebb281ae463c: Bootstrap starting.
I20260806 22:46:57.703106 9536 tablet_bootstrap.cc:654] T 2740dde8504b47ca9a96707481cce2da P a3ae77f560314bb69116291d0eb3f635: Neither blocks nor log segments found. Creating new log.
I20260806 22:46:57.703402 9534 tablet_bootstrap.cc:654] T 2740dde8504b47ca9a96707481cce2da P 4d6b767d87784ec3a9f8ebb281ae463c: Neither blocks nor log segments found. Creating new log.
I20260806 22:46:57.704039 9533 tablet_bootstrap.cc:492] T 2740dde8504b47ca9a96707481cce2da P 982377b0de554faf89f0dbff3b55a8d0: No bootstrap required, opened a new log
I20260806 22:46:57.704113 9533 ts_tablet_manager.cc:1397] T 2740dde8504b47ca9a96707481cce2da P 982377b0de554faf89f0dbff3b55a8d0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:46:57.704252 9533 raft_consensus.cc:348] T 2740dde8504b47ca9a96707481cce2da P 982377b0de554faf89f0dbff3b55a8d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d6b767d87784ec3a9f8ebb281ae463c" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 33289 } } peers { permanent_uuid: "a3ae77f560314bb69116291d0eb3f635" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 46183 } } peers { permanent_uuid: "982377b0de554faf89f0dbff3b55a8d0" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 38615 } }
I20260806 22:46:57.704361 9533 raft_consensus.cc:374] T 2740dde8504b47ca9a96707481cce2da P 982377b0de554faf89f0dbff3b55a8d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:46:57.704394 9533 raft_consensus.cc:729] T 2740dde8504b47ca9a96707481cce2da P 982377b0de554faf89f0dbff3b55a8d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 982377b0de554faf89f0dbff3b55a8d0, State: Initialized, Role: FOLLOWER
I20260806 22:46:57.704422 9536 tablet_bootstrap.cc:492] T 2740dde8504b47ca9a96707481cce2da P a3ae77f560314bb69116291d0eb3f635: No bootstrap required, opened a new log
I20260806 22:46:57.704469 9536 ts_tablet_manager.cc:1397] T 2740dde8504b47ca9a96707481cce2da P a3ae77f560314bb69116291d0eb3f635: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:46:57.704450 9533 consensus_queue.cc:260] T 2740dde8504b47ca9a96707481cce2da P 982377b0de554faf89f0dbff3b55a8d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d6b767d87784ec3a9f8ebb281ae463c" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 33289 } } peers { permanent_uuid: "a3ae77f560314bb69116291d0eb3f635" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 46183 } } peers { permanent_uuid: "982377b0de554faf89f0dbff3b55a8d0" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 38615 } }
I20260806 22:46:57.704582 9533 ts_tablet_manager.cc:1428] T 2740dde8504b47ca9a96707481cce2da P 982377b0de554faf89f0dbff3b55a8d0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:57.704919 9534 tablet_bootstrap.cc:492] T 2740dde8504b47ca9a96707481cce2da P 4d6b767d87784ec3a9f8ebb281ae463c: No bootstrap required, opened a new log
I20260806 22:46:57.704980 9534 ts_tablet_manager.cc:1397] T 2740dde8504b47ca9a96707481cce2da P 4d6b767d87784ec3a9f8ebb281ae463c: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 22:46:57.705001 9536 raft_consensus.cc:348] T 2740dde8504b47ca9a96707481cce2da P a3ae77f560314bb69116291d0eb3f635 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d6b767d87784ec3a9f8ebb281ae463c" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 33289 } } peers { permanent_uuid: "a3ae77f560314bb69116291d0eb3f635" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 46183 } } peers { permanent_uuid: "982377b0de554faf89f0dbff3b55a8d0" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 38615 } }
I20260806 22:46:57.705111 9536 raft_consensus.cc:374] T 2740dde8504b47ca9a96707481cce2da P a3ae77f560314bb69116291d0eb3f635 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:46:57.705137 9536 raft_consensus.cc:729] T 2740dde8504b47ca9a96707481cce2da P a3ae77f560314bb69116291d0eb3f635 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3ae77f560314bb69116291d0eb3f635, State: Initialized, Role: FOLLOWER
I20260806 22:46:57.705117 9534 raft_consensus.cc:348] T 2740dde8504b47ca9a96707481cce2da P 4d6b767d87784ec3a9f8ebb281ae463c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d6b767d87784ec3a9f8ebb281ae463c" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 33289 } } peers { permanent_uuid: "a3ae77f560314bb69116291d0eb3f635" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 46183 } } peers { permanent_uuid: "982377b0de554faf89f0dbff3b55a8d0" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 38615 } }
I20260806 22:46:57.705188 9534 raft_consensus.cc:374] T 2740dde8504b47ca9a96707481cce2da P 4d6b767d87784ec3a9f8ebb281ae463c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:46:57.705209 9534 raft_consensus.cc:729] T 2740dde8504b47ca9a96707481cce2da P 4d6b767d87784ec3a9f8ebb281ae463c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d6b767d87784ec3a9f8ebb281ae463c, State: Initialized, Role: FOLLOWER
I20260806 22:46:57.705189 9536 consensus_queue.cc:260] T 2740dde8504b47ca9a96707481cce2da P a3ae77f560314bb69116291d0eb3f635 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d6b767d87784ec3a9f8ebb281ae463c" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 33289 } } peers { permanent_uuid: "a3ae77f560314bb69116291d0eb3f635" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 46183 } } peers { permanent_uuid: "982377b0de554faf89f0dbff3b55a8d0" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 38615 } }
I20260806 22:46:57.705255 9534 consensus_queue.cc:260] T 2740dde8504b47ca9a96707481cce2da P 4d6b767d87784ec3a9f8ebb281ae463c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d6b767d87784ec3a9f8ebb281ae463c" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 33289 } } peers { permanent_uuid: "a3ae77f560314bb69116291d0eb3f635" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 46183 } } peers { permanent_uuid: "982377b0de554faf89f0dbff3b55a8d0" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 38615 } }
I20260806 22:46:57.705364 9534 ts_tablet_manager.cc:1428] T 2740dde8504b47ca9a96707481cce2da P 4d6b767d87784ec3a9f8ebb281ae463c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:57.705379 9536 ts_tablet_manager.cc:1428] T 2740dde8504b47ca9a96707481cce2da P a3ae77f560314bb69116291d0eb3f635: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260806 22:46:57.854060 9501 raft_consensus.cc:482] T 2740dde8504b47ca9a96707481cce2da P 4d6b767d87784ec3a9f8ebb281ae463c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:46:57.854173 9501 raft_consensus.cc:504] T 2740dde8504b47ca9a96707481cce2da P 4d6b767d87784ec3a9f8ebb281ae463c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d6b767d87784ec3a9f8ebb281ae463c" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 33289 } } peers { permanent_uuid: "a3ae77f560314bb69116291d0eb3f635" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 46183 } } peers { permanent_uuid: "982377b0de554faf89f0dbff3b55a8d0" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 38615 } }
I20260806 22:46:57.854370 9501 leader_election.cc:288] T 2740dde8504b47ca9a96707481cce2da P 4d6b767d87784ec3a9f8ebb281ae463c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3ae77f560314bb69116291d0eb3f635 (127.6.40.66:46183), 982377b0de554faf89f0dbff3b55a8d0 (127.6.40.67:38615)
I20260806 22:46:57.854591 9125 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2740dde8504b47ca9a96707481cce2da" candidate_uuid: "4d6b767d87784ec3a9f8ebb281ae463c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3ae77f560314bb69116291d0eb3f635" is_pre_election: true
I20260806 22:46:57.854633 9320 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2740dde8504b47ca9a96707481cce2da" candidate_uuid: "4d6b767d87784ec3a9f8ebb281ae463c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "982377b0de554faf89f0dbff3b55a8d0" is_pre_election: true
I20260806 22:46:57.854724 9320 raft_consensus.cc:2454] T 2740dde8504b47ca9a96707481cce2da P 982377b0de554faf89f0dbff3b55a8d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d6b767d87784ec3a9f8ebb281ae463c in term 0.
I20260806 22:46:57.854728 9125 raft_consensus.cc:2454] T 2740dde8504b47ca9a96707481cce2da P a3ae77f560314bb69116291d0eb3f635 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d6b767d87784ec3a9f8ebb281ae463c in term 0.
I20260806 22:46:57.854986 8838 leader_election.cc:302] T 2740dde8504b47ca9a96707481cce2da P 4d6b767d87784ec3a9f8ebb281ae463c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d6b767d87784ec3a9f8ebb281ae463c, a3ae77f560314bb69116291d0eb3f635; no voters:
I20260806 22:46:57.855129 9501 raft_consensus.cc:2781] T 2740dde8504b47ca9a96707481cce2da P 4d6b767d87784ec3a9f8ebb281ae463c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:46:57.855201 9501 raft_consensus.cc:482] T 2740dde8504b47ca9a96707481cce2da P 4d6b767d87784ec3a9f8ebb281ae463c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:46:57.855232 9501 raft_consensus.cc:3037] T 2740dde8504b47ca9a96707481cce2da P 4d6b767d87784ec3a9f8ebb281ae463c [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:46:57.856014 9501 raft_consensus.cc:504] T 2740dde8504b47ca9a96707481cce2da P 4d6b767d87784ec3a9f8ebb281ae463c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d6b767d87784ec3a9f8ebb281ae463c" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 33289 } } peers { permanent_uuid: "a3ae77f560314bb69116291d0eb3f635" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 46183 } } peers { permanent_uuid: "982377b0de554faf89f0dbff3b55a8d0" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 38615 } }
I20260806 22:46:57.856169 9501 leader_election.cc:288] T 2740dde8504b47ca9a96707481cce2da P 4d6b767d87784ec3a9f8ebb281ae463c [CANDIDATE]: Term 1 election: Requested vote from peers a3ae77f560314bb69116291d0eb3f635 (127.6.40.66:46183), 982377b0de554faf89f0dbff3b55a8d0 (127.6.40.67:38615)
I20260806 22:46:57.856389 9320 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2740dde8504b47ca9a96707481cce2da" candidate_uuid: "4d6b767d87784ec3a9f8ebb281ae463c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "982377b0de554faf89f0dbff3b55a8d0"
I20260806 22:46:57.856383 9125 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2740dde8504b47ca9a96707481cce2da" candidate_uuid: "4d6b767d87784ec3a9f8ebb281ae463c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3ae77f560314bb69116291d0eb3f635"
I20260806 22:46:57.856477 9125 raft_consensus.cc:3037] T 2740dde8504b47ca9a96707481cce2da P a3ae77f560314bb69116291d0eb3f635 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:46:57.856477 9320 raft_consensus.cc:3037] T 2740dde8504b47ca9a96707481cce2da P 982377b0de554faf89f0dbff3b55a8d0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:46:57.857362 9320 raft_consensus.cc:2454] T 2740dde8504b47ca9a96707481cce2da P 982377b0de554faf89f0dbff3b55a8d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d6b767d87784ec3a9f8ebb281ae463c in term 1.
I20260806 22:46:57.857363 9125 raft_consensus.cc:2454] T 2740dde8504b47ca9a96707481cce2da P a3ae77f560314bb69116291d0eb3f635 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d6b767d87784ec3a9f8ebb281ae463c in term 1.
I20260806 22:46:57.857570 8838 leader_election.cc:302] T 2740dde8504b47ca9a96707481cce2da P 4d6b767d87784ec3a9f8ebb281ae463c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d6b767d87784ec3a9f8ebb281ae463c, a3ae77f560314bb69116291d0eb3f635; no voters:
I20260806 22:46:57.857738 9501 raft_consensus.cc:2781] T 2740dde8504b47ca9a96707481cce2da P 4d6b767d87784ec3a9f8ebb281ae463c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:46:57.857821 9501 raft_consensus.cc:686] T 2740dde8504b47ca9a96707481cce2da P 4d6b767d87784ec3a9f8ebb281ae463c [term 1 LEADER]: Becoming Leader. State: Replica: 4d6b767d87784ec3a9f8ebb281ae463c, State: Running, Role: LEADER
I20260806 22:46:57.857877 9501 consensus_queue.cc:237] T 2740dde8504b47ca9a96707481cce2da P 4d6b767d87784ec3a9f8ebb281ae463c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d6b767d87784ec3a9f8ebb281ae463c" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 33289 } } peers { permanent_uuid: "a3ae77f560314bb69116291d0eb3f635" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 46183 } } peers { permanent_uuid: "982377b0de554faf89f0dbff3b55a8d0" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 38615 } }
I20260806 22:46:57.858548 8713 catalog_manager.cc:5495] T 2740dde8504b47ca9a96707481cce2da P 4d6b767d87784ec3a9f8ebb281ae463c reported cstate change: term changed from 0 to 1, leader changed from <none> to 4d6b767d87784ec3a9f8ebb281ae463c (127.6.40.65). New cstate: current_term: 1 leader_uuid: "4d6b767d87784ec3a9f8ebb281ae463c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d6b767d87784ec3a9f8ebb281ae463c" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 33289 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a3ae77f560314bb69116291d0eb3f635" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 46183 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "982377b0de554faf89f0dbff3b55a8d0" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 38615 } health_report { overall_health: UNKNOWN } } }
I20260806 22:46:57.892961 9546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:57.890401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33828 (local address 127.6.40.65:33289)
0806 22:46:57.890470 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:57.890472 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:57.890485 (+ 13us) server_negotiation.cc:407] Connection header received
0806 22:46:57.890561 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:57.890562 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:57.890573 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:57.890631 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:46:57.890771 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:57.891424 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:57.892181 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:57.892451 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:57.892512 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:57.892687 (+ 175us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:46:57.892725 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:46:57.892774 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:57.892827 (+ 53us) server_negotiation.cc:299] Negotiation successful
0806 22:46:57.892870 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 22:46:57.895434 9320 raft_consensus.cc:1261] T 2740dde8504b47ca9a96707481cce2da P 982377b0de554faf89f0dbff3b55a8d0 [term 1 FOLLOWER]: Refusing update from remote peer 4d6b767d87784ec3a9f8ebb281ae463c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:46:57.895471 9125 raft_consensus.cc:1261] T 2740dde8504b47ca9a96707481cce2da P a3ae77f560314bb69116291d0eb3f635 [term 1 FOLLOWER]: Refusing update from remote peer 4d6b767d87784ec3a9f8ebb281ae463c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:46:57.895694 9424 consensus_queue.cc:1035] T 2740dde8504b47ca9a96707481cce2da P 4d6b767d87784ec3a9f8ebb281ae463c [LEADER]: Connected to new peer: Peer: permanent_uuid: "982377b0de554faf89f0dbff3b55a8d0" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 38615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:46:57.895881 9501 consensus_queue.cc:1035] T 2740dde8504b47ca9a96707481cce2da P 4d6b767d87784ec3a9f8ebb281ae463c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3ae77f560314bb69116291d0eb3f635" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 46183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 22:46:57.900498 9489 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260806 22:46:59.418608 8713 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40584:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:46:59.418715 8713 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40584:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:46:59.419669 8713 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9d0e72021aa443289c0bb45de789d382: Sending DeleteTablet for 3 replicas of tablet 2740dde8504b47ca9a96707481cce2da
I20260806 22:46:59.419991 8902 tablet_service.cc:1514] Processing DeleteTablet for tablet 2740dde8504b47ca9a96707481cce2da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:46:59 UTC) from {username='kudu', principal='kudu/127.6.40.126@KRBTEST.COM'} at 127.0.0.1:33792
I20260806 22:46:59.419991 9293 tablet_service.cc:1514] Processing DeleteTablet for tablet 2740dde8504b47ca9a96707481cce2da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:46:59 UTC) from {username='kudu', principal='kudu/127.6.40.126@KRBTEST.COM'} at 127.0.0.1:57548
I20260806 22:46:59.420047 9098 tablet_service.cc:1514] Processing DeleteTablet for tablet 2740dde8504b47ca9a96707481cce2da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:46:59 UTC) from {username='kudu', principal='kudu/127.6.40.126@KRBTEST.COM'} at 127.0.0.1:51600
I20260806 22:46:59.420145 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 8816
I20260806 22:46:59.431322 9596 tablet_replica.cc:330] T 2740dde8504b47ca9a96707481cce2da P a3ae77f560314bb69116291d0eb3f635: stopping tablet replica
I20260806 22:46:59.431442 9596 raft_consensus.cc:2229] T 2740dde8504b47ca9a96707481cce2da P a3ae77f560314bb69116291d0eb3f635 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:46:59.431497 9596 raft_consensus.cc:2258] T 2740dde8504b47ca9a96707481cce2da P a3ae77f560314bb69116291d0eb3f635 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:46:59.431838 9596 ts_tablet_manager.cc:1905] T 2740dde8504b47ca9a96707481cce2da P a3ae77f560314bb69116291d0eb3f635: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:46:59.432942 9594 tablet_replica.cc:330] T 2740dde8504b47ca9a96707481cce2da P 982377b0de554faf89f0dbff3b55a8d0: stopping tablet replica
I20260806 22:46:59.433063 9594 raft_consensus.cc:2229] T 2740dde8504b47ca9a96707481cce2da P 982377b0de554faf89f0dbff3b55a8d0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:46:59.433122 9594 raft_consensus.cc:2258] T 2740dde8504b47ca9a96707481cce2da P 982377b0de554faf89f0dbff3b55a8d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:46:59.433389 9594 ts_tablet_manager.cc:1905] T 2740dde8504b47ca9a96707481cce2da P 982377b0de554faf89f0dbff3b55a8d0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:46:59.434228 9596 ts_tablet_manager.cc:1918] T 2740dde8504b47ca9a96707481cce2da P a3ae77f560314bb69116291d0eb3f635: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 22:46:59.434305 9596 log.cc:1196] T 2740dde8504b47ca9a96707481cce2da P a3ae77f560314bb69116291d0eb3f635: Deleting WAL directory at /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/minicluster-data/ts-1/wal/wals/2740dde8504b47ca9a96707481cce2da
I20260806 22:46:59.434535 9596 ts_tablet_manager.cc:1939] T 2740dde8504b47ca9a96707481cce2da P a3ae77f560314bb69116291d0eb3f635: Deleting consensus metadata
I20260806 22:46:59.434900 9594 ts_tablet_manager.cc:1918] T 2740dde8504b47ca9a96707481cce2da P 982377b0de554faf89f0dbff3b55a8d0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 22:46:59.434979 9594 log.cc:1196] T 2740dde8504b47ca9a96707481cce2da P 982377b0de554faf89f0dbff3b55a8d0: Deleting WAL directory at /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056405724609-6305-0/minicluster-data/ts-2/wal/wals/2740dde8504b47ca9a96707481cce2da
I20260806 22:46:59.435154 8696 catalog_manager.cc:4841] TS a3ae77f560314bb69116291d0eb3f635 (127.6.40.66:46183): tablet 2740dde8504b47ca9a96707481cce2da (table test-table [id=b8f50e53088047899afb7085a24a89b9]) successfully deleted
I20260806 22:46:59.435264 9594 ts_tablet_manager.cc:1939] T 2740dde8504b47ca9a96707481cce2da P 982377b0de554faf89f0dbff3b55a8d0: Deleting consensus metadata
I20260806 22:46:59.435638 8697 catalog_manager.cc:4841] TS 982377b0de554faf89f0dbff3b55a8d0 (127.6.40.67:38615): tablet 2740dde8504b47ca9a96707481cce2da (table test-table [id=b8f50e53088047899afb7085a24a89b9]) successfully deleted
W20260806 22:46:59.438333 8698 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.40.65:33289 (error 108)
I20260806 22:46:59.438616 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 9002
I20260806 22:46:59.445721 9598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:59.440579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.65:33289 (local address 127.0.0.1:33840)
0806 22:46:59.445614 (+ 5035us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:59.445677 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.65:33289: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":5006,"thread_start_us":1011,"threads_started":1}
W20260806 22:46:59.445883 8698 catalog_manager.cc:4584] TS 4d6b767d87784ec3a9f8ebb281ae463c (127.6.40.65:33289): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2740dde8504b47ca9a96707481cce2da: Network error: Client connection negotiation failed: client connection to 127.6.40.65:33289: connect: Connection refused (error 111)
I20260806 22:46:59.454279 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 9194
I20260806 22:46:59.466840 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 8669
2026-08-06T22:46:59Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6122 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:46:59 dist-test-slave-zh2d krb5kdc[9604](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:46:59 dist-test-slave-zh2d krb5kdc[9604](info): set up 2 sockets
Aug 06 22:46:59 dist-test-slave-zh2d krb5kdc[9604](info): commencing operation
krb5kdc: starting...
W20260806 22:47:00.736112 6305 mini_kdc.cc:121] Time spent starting KDC: real 1.243s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/krb5kdc/test-user.keytab.
Aug 06 22:47:00 dist-test-slave-zh2d krb5kdc[9604](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056420, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:47:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:47:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.40.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.40.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.40.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab.
Entry for principal kudu/127.6.40.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab.
I20260806 22:47:00.864609 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.126:42441
--webserver_interface=127.6.40.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab
--principal=kudu/127.6.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:41259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.40.126:42441
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:47:00.948963 9623 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:47:00.949175 9623 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:47:00.949199 9623 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:47:00.950384 9623 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:47:00.950428 9623 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:47:00.950443 9623 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:47:00.950456 9623 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:47:00.950522 9623 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:47:00.952024 9623 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:41259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.40.126:42441
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab
--principal=kudu/127.6.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.126:42441
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9623
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:47:00.952291 9623 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:47:00.952548 9623 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:47:00.957953 9630 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:00.957999 9623 server_base.cc:1034] running on GCE node
W20260806 22:47:00.957948 9633 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:00.957950 9629 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:00.958396 9623 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:47:00.958731 9623 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:47:00.959882 9623 hybrid_clock.cc:648] HybridClock initialized: now 1786056420959858 us; error 46 us; skew 500 ppm
Aug 06 22:47:00 dist-test-slave-zh2d krb5kdc[9604](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056420, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:47:00.962478 9623 init.cc:377] Logged in from keytab as kudu/127.6.40.126@KRBTEST.COM (short username kudu)
I20260806 22:47:00.962970 9623 webserver.cc:460] Webserver started at http://127.6.40.126:35791/ using document root <none> and password file <none>
I20260806 22:47:00.963236 9623 fs_manager.cc:359] Metadata directory not provided
I20260806 22:47:00.963285 9623 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:47:00.963480 9623 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:47:00.964562 9623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/minicluster-data/master-0/data/instance:
uuid: "3c5d8b4fd99e49afba9e491b20affdfb"
format_stamp: "Formatted at 2026-08-06 22:47:00 on dist-test-slave-zh2d"
server_key: "a369dc12351198ee62e97bfac1f74a19"
server_key_iv: "e8953a99006b026dd78c984f82a93d62"
server_key_version: "encryptionkey@0"
I20260806 22:47:00.964968 9623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/minicluster-data/master-0/wal/instance:
uuid: "3c5d8b4fd99e49afba9e491b20affdfb"
format_stamp: "Formatted at 2026-08-06 22:47:00 on dist-test-slave-zh2d"
server_key: "a369dc12351198ee62e97bfac1f74a19"
server_key_iv: "e8953a99006b026dd78c984f82a93d62"
server_key_version: "encryptionkey@0"
I20260806 22:47:00.966360 9623 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:47:00.967080 9641 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:00.967255 9623 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:47:00.967366 9623 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/minicluster-data/master-0/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/minicluster-data/master-0/wal
uuid: "3c5d8b4fd99e49afba9e491b20affdfb"
format_stamp: "Formatted at 2026-08-06 22:47:00 on dist-test-slave-zh2d"
server_key: "a369dc12351198ee62e97bfac1f74a19"
server_key_iv: "e8953a99006b026dd78c984f82a93d62"
server_key_version: "encryptionkey@0"
I20260806 22:47:00.967439 9623 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:47:00.991868 9623 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:47:00.993943 9623 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:47:00.994130 9623 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:47:00.998399 9623 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.126:42441
I20260806 22:47:00.998458 9712 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.126:42441 every 8 connection(s)
I20260806 22:47:00.998844 9623 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
I20260806 22:47:00.999753 9713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:47:01.000140 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 9623
I20260806 22:47:01.000259 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/minicluster-data/master-0/wal/instance
I20260806 22:47:01.000466 6305 external_mini_cluster.cc:1428] Setting key 8943f6381f3bb2c448c351d0ebdd6033
I20260806 22:47:01.002985 9713 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c5d8b4fd99e49afba9e491b20affdfb: Bootstrap starting.
I20260806 22:47:01.003693 9713 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3c5d8b4fd99e49afba9e491b20affdfb: Neither blocks nor log segments found. Creating new log.
I20260806 22:47:01.004032 9713 log.cc:824] T 00000000000000000000000000000000 P 3c5d8b4fd99e49afba9e491b20affdfb: Log is configured to *not* fsync() on all Append() calls
I20260806 22:47:01.004928 9713 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c5d8b4fd99e49afba9e491b20affdfb: No bootstrap required, opened a new log
I20260806 22:47:01.007202 9713 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3c5d8b4fd99e49afba9e491b20affdfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c5d8b4fd99e49afba9e491b20affdfb" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 42441 } }
I20260806 22:47:01.007381 9713 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3c5d8b4fd99e49afba9e491b20affdfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:47:01.007439 9713 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3c5d8b4fd99e49afba9e491b20affdfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c5d8b4fd99e49afba9e491b20affdfb, State: Initialized, Role: FOLLOWER
I20260806 22:47:01.007576 9713 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3c5d8b4fd99e49afba9e491b20affdfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c5d8b4fd99e49afba9e491b20affdfb" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 42441 } }
I20260806 22:47:01.007674 9713 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3c5d8b4fd99e49afba9e491b20affdfb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:47:01.007721 9713 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3c5d8b4fd99e49afba9e491b20affdfb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:47:01.007776 9713 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3c5d8b4fd99e49afba9e491b20affdfb [term 0 FOLLOWER]: Advancing to term 1
Aug 06 22:47:01 dist-test-slave-zh2d krb5kdc[9604](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056420, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
I20260806 22:47:01.008438 9713 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3c5d8b4fd99e49afba9e491b20affdfb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c5d8b4fd99e49afba9e491b20affdfb" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 42441 } }
I20260806 22:47:01.008580 9713 leader_election.cc:302] T 00000000000000000000000000000000 P 3c5d8b4fd99e49afba9e491b20affdfb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3c5d8b4fd99e49afba9e491b20affdfb; no voters:
I20260806 22:47:01.008785 9713 leader_election.cc:288] T 00000000000000000000000000000000 P 3c5d8b4fd99e49afba9e491b20affdfb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:47:01.008819 9721 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3c5d8b4fd99e49afba9e491b20affdfb [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:47:01.009042 9721 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3c5d8b4fd99e49afba9e491b20affdfb [term 1 LEADER]: Becoming Leader. State: Replica: 3c5d8b4fd99e49afba9e491b20affdfb, State: Running, Role: LEADER
I20260806 22:47:01.009102 9713 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3c5d8b4fd99e49afba9e491b20affdfb [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:47:01.009158 9721 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3c5d8b4fd99e49afba9e491b20affdfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c5d8b4fd99e49afba9e491b20affdfb" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 42441 } }
I20260806 22:47:01.009771 9723 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c5d8b4fd99e49afba9e491b20affdfb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3c5d8b4fd99e49afba9e491b20affdfb. Latest consensus state: current_term: 1 leader_uuid: "3c5d8b4fd99e49afba9e491b20affdfb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c5d8b4fd99e49afba9e491b20affdfb" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 42441 } } }
I20260806 22:47:01.010851 9723 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c5d8b4fd99e49afba9e491b20affdfb [sys.catalog]: This master's current role is: LEADER
I20260806 22:47:01.009666 9722 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c5d8b4fd99e49afba9e491b20affdfb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3c5d8b4fd99e49afba9e491b20affdfb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c5d8b4fd99e49afba9e491b20affdfb" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 42441 } } }
I20260806 22:47:01.011085 9722 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c5d8b4fd99e49afba9e491b20affdfb [sys.catalog]: This master's current role is: LEADER
W20260806 22:47:01.011412 9740 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3c5d8b4fd99e49afba9e491b20affdfb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:47:01.011487 9740 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:47:01.011554 9742 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:47:01.011785 9742 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:47:01.012732 9718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:01.001086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38280 (local address 127.6.40.126:42441)
0806 22:47:01.001315 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:01.001325 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:01.001363 (+ 38us) server_negotiation.cc:407] Connection header received
0806 22:47:01.001930 (+ 567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:01.001939 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:01.002004 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:01.002165 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:01.002843 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:01.004243 (+ 1400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:01.005219 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:01.005822 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:01.005983 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:01.008532 (+ 2549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:01.008537 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:01.008537 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:01.008550 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:01.010956 (+ 2406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:01.011385 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:01.011389 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:01.011389 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:01.011551 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:01.011785 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:01.011786 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:01.011786 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:01.012106 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:01.012273 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:01.012380 (+ 107us) server_negotiation.cc:299] Negotiation successful
0806 22:47:01.012531 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":104,"threads_started":1}
I20260806 22:47:01.013401 9742 catalog_manager.cc:1347] Generated new cluster ID: ca85433a978e45ef8d157c79fcbf5f4d
I20260806 22:47:01.013454 9742 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:47:01.021718 9742 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:47:01.022200 9742 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:47:01.034202 9742 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3c5d8b4fd99e49afba9e491b20affdfb: Generated new TSK 0
I20260806 22:47:01.034478 9742 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.40.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.40.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.40.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.65.keytab.
Entry for principal kudu/127.6.40.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.65.keytab.
I20260806 22:47:01.063972 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.65:0
--local_ip_for_outbound_sockets=127.6.40.65
--webserver_interface=127.6.40.65
--webserver_port=0
--tserver_master_addrs=127.6.40.126:42441
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.65.keytab
--principal=kudu/127.6.40.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:41259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:47:01.149113 9748 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:47:01.149334 9748 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:47:01.149358 9748 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:47:01.150511 9748 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:47:01.150556 9748 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:47:01.150583 9748 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.65
I20260806 22:47:01.151991 9748 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:41259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.65.keytab
--principal=kudu/127.6.40.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.40.65
--webserver_port=0
--tserver_master_addrs=127.6.40.126:42441
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9748
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.65
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:47:01.152230 9748 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:47:01.152482 9748 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:47:01.157642 9759 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:01.157655 9761 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:01.157647 9756 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:01.157732 9748 server_base.cc:1034] running on GCE node
I20260806 22:47:01.158061 9748 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:47:01.158298 9748 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:47:01.159461 9748 hybrid_clock.cc:648] HybridClock initialized: now 1786056421159467 us; error 51 us; skew 500 ppm
Aug 06 22:47:01 dist-test-slave-zh2d krb5kdc[9604](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056421, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:47:01.162032 9748 init.cc:377] Logged in from keytab as kudu/127.6.40.65@KRBTEST.COM (short username kudu)
I20260806 22:47:01.162537 9748 webserver.cc:460] Webserver started at http://127.6.40.65:32953/ using document root <none> and password file <none>
I20260806 22:47:01.162767 9748 fs_manager.cc:359] Metadata directory not provided
I20260806 22:47:01.162835 9748 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:47:01.162973 9748 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:47:01.164155 9748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-0/data/instance:
uuid: "a3ea8a71609a4b7796703058b395bbdc"
format_stamp: "Formatted at 2026-08-06 22:47:01 on dist-test-slave-zh2d"
server_key: "21626325f2c402136f1e778ef19be1d5"
server_key_iv: "b4bff43013c1e045cf8abe4f53ed837f"
server_key_version: "encryptionkey@0"
I20260806 22:47:01.164533 9748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-0/wal/instance:
uuid: "a3ea8a71609a4b7796703058b395bbdc"
format_stamp: "Formatted at 2026-08-06 22:47:01 on dist-test-slave-zh2d"
server_key: "21626325f2c402136f1e778ef19be1d5"
server_key_iv: "b4bff43013c1e045cf8abe4f53ed837f"
server_key_version: "encryptionkey@0"
I20260806 22:47:01.165949 9748 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:47:01.166695 9771 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:01.166868 9748 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:47:01.166977 9748 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-0/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-0/wal
uuid: "a3ea8a71609a4b7796703058b395bbdc"
format_stamp: "Formatted at 2026-08-06 22:47:01 on dist-test-slave-zh2d"
server_key: "21626325f2c402136f1e778ef19be1d5"
server_key_iv: "b4bff43013c1e045cf8abe4f53ed837f"
server_key_version: "encryptionkey@0"
I20260806 22:47:01.167054 9748 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:47:01.189252 9748 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:47:01.191741 9748 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:47:01.191944 9748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:47:01.192226 9748 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:47:01.192559 9748 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:47:01.192615 9748 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:01.192657 9748 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:47:01.192708 9748 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:01.199389 9748 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.65:40451
I20260806 22:47:01.199494 9945 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.65:40451 every 8 connection(s)
I20260806 22:47:01.199854 9748 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
Aug 06 22:47:01 dist-test-slave-zh2d krb5kdc[9604](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056421, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.65@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
I20260806 22:47:01.209512 9718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:01.200623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.65:53937 (local address 127.6.40.126:42441)
0806 22:47:01.200670 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:01.200672 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:01.200956 (+ 284us) server_negotiation.cc:407] Connection header received
0806 22:47:01.201531 (+ 575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:01.201533 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:01.201546 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:01.201614 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:01.202613 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:01.203148 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:01.204169 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:01.204400 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:01.204503 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:01.206760 (+ 2257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:01.206761 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:01.206762 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:01.206769 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:01.208037 (+ 1268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:01.208440 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:01.208441 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:01.208441 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:01.208507 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:01.208778 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:01.208779 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:01.208780 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:01.208938 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:01.209035 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:01.209326 (+ 291us) server_negotiation.cc:299] Negotiation successful
0806 22:47:01.209418 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 22:47:01.209748 9954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:01.200612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:42441 (local address 127.6.40.65:53937)
0806 22:47:01.200804 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:01.200864 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:01.201412 (+ 548us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:01.201685 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:01.201689 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:01.202297 (+ 608us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:01.202503 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:01.202508 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:01.203260 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:01.203261 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:01.204071 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:01.204074 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:01.204519 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:01.204520 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:01.204649 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:01.205229 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:01.205237 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:01.206650 (+ 1413us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:47:01.208126 (+ 1476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:01.208135 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:01.208137 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:01.208360 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:01.208593 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:01.208594 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:01.208595 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:01.208707 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:01.209027 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:47:01.209035 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:47:01.209190 (+ 155us) client_negotiation.cc:769] Sending connection context
0806 22:47:01.209303 (+ 113us) client_negotiation.cc:240] Negotiation successful
0806 22:47:01.209466 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":96,"threads_started":1}
I20260806 22:47:01.209928 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 9748
I20260806 22:47:01.210021 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-0/wal/instance
I20260806 22:47:01.210201 6305 external_mini_cluster.cc:1428] Setting key 0b48490fd8ee283945345da4dbb1cbff
I20260806 22:47:01.210268 9947 heartbeater.cc:347] Connected to a master server at 127.6.40.126:42441
I20260806 22:47:01.210361 9947 heartbeater.cc:464] Registering TS with master...
I20260806 22:47:01.210566 9947 heartbeater.cc:511] Master 127.6.40.126:42441 requested a full tablet report, sending...
I20260806 22:47:01.211103 9668 ts_manager.cc:194] Registered new tserver with Master: a3ea8a71609a4b7796703058b395bbdc (127.6.40.65:40451)
I20260806 22:47:01.211715 9668 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.40.65@KRBTEST.COM'} at 127.6.40.65:53937
WARNING: no policy specified for kudu/127.6.40.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.40.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.40.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.66.keytab.
Entry for principal kudu/127.6.40.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.66.keytab.
I20260806 22:47:01.236138 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.66:0
--local_ip_for_outbound_sockets=127.6.40.66
--webserver_interface=127.6.40.66
--webserver_port=0
--tserver_master_addrs=127.6.40.126:42441
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.66.keytab
--principal=kudu/127.6.40.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:41259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:47:01.319995 9961 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:47:01.320194 9961 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:47:01.320217 9961 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:47:01.321502 9961 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:47:01.321544 9961 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:47:01.321570 9961 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.66
I20260806 22:47:01.323060 9961 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:41259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.66.keytab
--principal=kudu/127.6.40.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.40.66
--webserver_port=0
--tserver_master_addrs=127.6.40.126:42441
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9961
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.66
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:47:01.323307 9961 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:47:01.323544 9961 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:47:01.328826 9972 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:01.328852 9966 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:01.328894 9970 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:01.329084 9961 server_base.cc:1034] running on GCE node
I20260806 22:47:01.329375 9961 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:47:01.329613 9961 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:47:01.330770 9961 hybrid_clock.cc:648] HybridClock initialized: now 1786056421330761 us; error 55 us; skew 500 ppm
Aug 06 22:47:01 dist-test-slave-zh2d krb5kdc[9604](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056421, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:47:01.333405 9961 init.cc:377] Logged in from keytab as kudu/127.6.40.66@KRBTEST.COM (short username kudu)
I20260806 22:47:01.333851 9961 webserver.cc:460] Webserver started at http://127.6.40.66:33225/ using document root <none> and password file <none>
I20260806 22:47:01.334106 9961 fs_manager.cc:359] Metadata directory not provided
I20260806 22:47:01.334179 9961 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:47:01.334329 9961 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:47:01.335371 9961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-1/data/instance:
uuid: "4d4ce85eb4ca47548374a73518601f2a"
format_stamp: "Formatted at 2026-08-06 22:47:01 on dist-test-slave-zh2d"
server_key: "ac82b58d0c3ff4b3a7e6f655ca700e70"
server_key_iv: "3d48d036f05b95aec0e3e648ea995720"
server_key_version: "encryptionkey@0"
I20260806 22:47:01.335736 9961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-1/wal/instance:
uuid: "4d4ce85eb4ca47548374a73518601f2a"
format_stamp: "Formatted at 2026-08-06 22:47:01 on dist-test-slave-zh2d"
server_key: "ac82b58d0c3ff4b3a7e6f655ca700e70"
server_key_iv: "3d48d036f05b95aec0e3e648ea995720"
server_key_version: "encryptionkey@0"
I20260806 22:47:01.337208 9961 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:47:01.337996 9984 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:01.338217 9961 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:47:01.338325 9961 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-1/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-1/wal
uuid: "4d4ce85eb4ca47548374a73518601f2a"
format_stamp: "Formatted at 2026-08-06 22:47:01 on dist-test-slave-zh2d"
server_key: "ac82b58d0c3ff4b3a7e6f655ca700e70"
server_key_iv: "3d48d036f05b95aec0e3e648ea995720"
server_key_version: "encryptionkey@0"
I20260806 22:47:01.338400 9961 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:47:01.358363 9961 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:47:01.360266 9961 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:47:01.360436 9961 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:47:01.360695 9961 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:47:01.361089 9961 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:47:01.361145 9961 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:01.361192 9961 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:47:01.361240 9961 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:01.367518 9961 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.66:46565
I20260806 22:47:01.367620 10144 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.66:46565 every 8 connection(s)
I20260806 22:47:01.367929 9961 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
I20260806 22:47:01.371694 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 9961
I20260806 22:47:01.371773 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-1/wal/instance
I20260806 22:47:01.371949 6305 external_mini_cluster.cc:1428] Setting key 86a89fa72615de998dccdc7fe05a245a
Aug 06 22:47:01 dist-test-slave-zh2d krb5kdc[9604](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056421, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.66@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
I20260806 22:47:01.378266 9718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:01.368743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.66:36345 (local address 127.6.40.126:42441)
0806 22:47:01.368821 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:01.368823 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:01.369041 (+ 218us) server_negotiation.cc:407] Connection header received
0806 22:47:01.369553 (+ 512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:01.369554 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:01.369582 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:01.369661 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:01.370774 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:01.371401 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:01.372622 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:01.372830 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:01.372942 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:01.375234 (+ 2292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:01.375237 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:01.375237 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:01.375247 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:01.376735 (+ 1488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:01.377269 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:01.377270 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:01.377270 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:01.377325 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:01.377618 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:01.377618 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:01.377619 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:01.377719 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:01.377771 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:01.378138 (+ 367us) server_negotiation.cc:299] Negotiation successful
0806 22:47:01.378201 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 22:47:01.378607 10149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:01.368709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:42441 (local address 127.6.40.66:36345)
0806 22:47:01.368889 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:01.368931 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:01.369446 (+ 515us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:01.369721 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:01.369725 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:01.370435 (+ 710us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:01.370692 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:01.370698 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:01.371500 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:01.371501 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:01.372519 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:01.372522 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:01.372956 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:01.372957 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:01.373108 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:01.373655 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:01.373664 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:01.375131 (+ 1467us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:47:01.376816 (+ 1685us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:01.376825 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:01.376828 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:01.377191 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:01.377397 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:01.377397 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:01.377398 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:01.377547 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:01.377789 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:47:01.377797 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:47:01.377981 (+ 184us) client_negotiation.cc:769] Sending connection context
0806 22:47:01.378121 (+ 140us) client_negotiation.cc:240] Negotiation successful
0806 22:47:01.378317 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":83,"threads_started":1}
I20260806 22:47:01.379192 10145 heartbeater.cc:347] Connected to a master server at 127.6.40.126:42441
I20260806 22:47:01.379287 10145 heartbeater.cc:464] Registering TS with master...
I20260806 22:47:01.379487 10145 heartbeater.cc:511] Master 127.6.40.126:42441 requested a full tablet report, sending...
I20260806 22:47:01.379873 9668 ts_manager.cc:194] Registered new tserver with Master: 4d4ce85eb4ca47548374a73518601f2a (127.6.40.66:46565)
I20260806 22:47:01.380200 9668 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.40.66@KRBTEST.COM'} at 127.6.40.66:36345
WARNING: no policy specified for kudu/127.6.40.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.40.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.40.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.67.keytab.
Entry for principal kudu/127.6.40.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.67.keytab.
I20260806 22:47:01.399358 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.67:0
--local_ip_for_outbound_sockets=127.6.40.67
--webserver_interface=127.6.40.67
--webserver_port=0
--tserver_master_addrs=127.6.40.126:42441
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.67.keytab
--principal=kudu/127.6.40.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:41259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:47:01.484791 10152 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:47:01.485033 10152 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:47:01.485061 10152 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:47:01.486267 10152 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:47:01.486310 10152 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:47:01.486337 10152 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.67
I20260806 22:47:01.487838 10152 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:41259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.67.keytab
--principal=kudu/127.6.40.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.40.67
--webserver_port=0
--tserver_master_addrs=127.6.40.126:42441
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10152
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.67
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:47:01.488082 10152 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:47:01.488343 10152 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:47:01.493695 10161 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:01.493698 10159 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:01.493728 10164 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:01.493731 10152 server_base.cc:1034] running on GCE node
I20260806 22:47:01.494185 10152 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:47:01.494467 10152 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:47:01.495638 10152 hybrid_clock.cc:648] HybridClock initialized: now 1786056421495617 us; error 34 us; skew 500 ppm
Aug 06 22:47:01 dist-test-slave-zh2d krb5kdc[9604](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056421, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:47:01.498236 10152 init.cc:377] Logged in from keytab as kudu/127.6.40.67@KRBTEST.COM (short username kudu)
I20260806 22:47:01.498724 10152 webserver.cc:460] Webserver started at http://127.6.40.67:45901/ using document root <none> and password file <none>
I20260806 22:47:01.498960 10152 fs_manager.cc:359] Metadata directory not provided
I20260806 22:47:01.499029 10152 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:47:01.499169 10152 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:47:01.500242 10152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-2/data/instance:
uuid: "d5ba49a0789741aba7f0621277236950"
format_stamp: "Formatted at 2026-08-06 22:47:01 on dist-test-slave-zh2d"
server_key: "b162a82dec64c38660b19c5320ec42e9"
server_key_iv: "fc9b90047af2d6e2b8acf223798e78f4"
server_key_version: "encryptionkey@0"
I20260806 22:47:01.500639 10152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-2/wal/instance:
uuid: "d5ba49a0789741aba7f0621277236950"
format_stamp: "Formatted at 2026-08-06 22:47:01 on dist-test-slave-zh2d"
server_key: "b162a82dec64c38660b19c5320ec42e9"
server_key_iv: "fc9b90047af2d6e2b8acf223798e78f4"
server_key_version: "encryptionkey@0"
I20260806 22:47:01.502139 10152 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:47:01.502907 10177 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:01.503083 10152 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:47:01.503188 10152 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-2/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-2/wal
uuid: "d5ba49a0789741aba7f0621277236950"
format_stamp: "Formatted at 2026-08-06 22:47:01 on dist-test-slave-zh2d"
server_key: "b162a82dec64c38660b19c5320ec42e9"
server_key_iv: "fc9b90047af2d6e2b8acf223798e78f4"
server_key_version: "encryptionkey@0"
I20260806 22:47:01.503271 10152 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:47:01.516299 10152 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:47:01.518589 10152 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:47:01.518754 10152 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:47:01.519011 10152 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:47:01.519335 10152 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:47:01.519389 10152 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:01.519433 10152 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:47:01.519462 10152 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:01.525926 10152 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.67:45671
I20260806 22:47:01.525980 10339 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.67:45671 every 8 connection(s)
I20260806 22:47:01.526305 10152 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
Aug 06 22:47:01 dist-test-slave-zh2d krb5kdc[9604](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056421, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.67@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
I20260806 22:47:01.534991 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 10152
I20260806 22:47:01.535091 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-2/wal/instance
I20260806 22:47:01.535266 6305 external_mini_cluster.cc:1428] Setting key 9b488207c64ee9ac4a9bb6790ac668c3
I20260806 22:47:01.535531 9718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:01.527053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.67:38709 (local address 127.6.40.126:42441)
0806 22:47:01.527095 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:01.527097 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:01.527319 (+ 222us) server_negotiation.cc:407] Connection header received
0806 22:47:01.527811 (+ 492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:01.527812 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:01.527823 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:01.527876 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:01.528792 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:01.529337 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:01.530323 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:01.530507 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:01.530563 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:01.532749 (+ 2186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:01.532750 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:01.532751 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:01.532758 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:01.534080 (+ 1322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:01.534514 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:01.534515 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:01.534515 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:01.534568 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:01.534827 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:01.534828 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:01.534828 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:01.534948 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:01.535049 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:01.535350 (+ 301us) server_negotiation.cc:299] Negotiation successful
0806 22:47:01.535438 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 22:47:01.535756 10343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:01.527044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:42441 (local address 127.6.40.67:38709)
0806 22:47:01.527193 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:01.527230 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:01.527728 (+ 498us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:01.527929 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:01.527932 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:01.528523 (+ 591us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:01.528716 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:01.528721 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:01.529421 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:01.529421 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:01.530244 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:01.530246 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:01.530587 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:01.530588 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:01.530687 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:01.531231 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:01.531239 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:01.532662 (+ 1423us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:47:01.534179 (+ 1517us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:01.534187 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:01.534190 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:01.534435 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:01.534632 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:01.534632 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:01.534633 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:01.534732 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:01.535024 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:47:01.535032 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:47:01.535212 (+ 180us) client_negotiation.cc:769] Sending connection context
0806 22:47:01.535324 (+ 112us) client_negotiation.cc:240] Negotiation successful
0806 22:47:01.535486 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":81,"threads_started":1}
I20260806 22:47:01.536312 10340 heartbeater.cc:347] Connected to a master server at 127.6.40.126:42441
I20260806 22:47:01.536419 10340 heartbeater.cc:464] Registering TS with master...
I20260806 22:47:01.536600 10340 heartbeater.cc:511] Master 127.6.40.126:42441 requested a full tablet report, sending...
I20260806 22:47:01.537014 9668 ts_manager.cc:194] Registered new tserver with Master: d5ba49a0789741aba7f0621277236950 (127.6.40.67:45671)
I20260806 22:47:01.537403 9668 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.40.67@KRBTEST.COM'} at 127.6.40.67:38709
I20260806 22:47:01.547689 6305 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 22:47:01.560971 9718 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:47:01.559876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38296 (local address 127.6.40.126:42441)
0806 22:47:01.559949 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:01.559951 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:01.560095 (+ 144us) server_negotiation.cc:407] Connection header received
0806 22:47:01.560183 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:01.560184 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:01.560200 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:01.560264 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:01.560829 (+ 565us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38296: BlockingRecv error: recv got EOF from 127.0.0.1:38296 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 22:47:01.561851 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 9748
I20260806 22:47:01.582413 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 9961
I20260806 22:47:01.590309 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 10152
I20260806 22:47:01.604470 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 9623
2026-08-06T22:47:01Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2134 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056405724609-6305-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:47:01 dist-test-slave-zh2d krb5kdc[10355](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:47:01 dist-test-slave-zh2d krb5kdc[10355](info): set up 2 sockets
Aug 06 22:47:01 dist-test-slave-zh2d krb5kdc[10355](info): commencing operation
krb5kdc: starting...
W20260806 22:47:02.850250 6305 mini_kdc.cc:121] Time spent starting KDC: real 1.222s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056405724609-6305-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056405724609-6305-0/krb5kdc/test-user.keytab.
Aug 06 22:47:02 dist-test-slave-zh2d krb5kdc[10355](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056422, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:47:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:47:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.40.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.40.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.40.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab.
Entry for principal kudu/127.6.40.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab.
I20260806 22:47:02.980230 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056405724609-6305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056405724609-6305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056405724609-6305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.126:37985
--webserver_interface=127.6.40.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab
--principal=kudu/127.6.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:45851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.40.126:37985
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:47:03.063916 10371 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:47:03.064131 10371 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:47:03.064154 10371 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:47:03.065356 10371 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:47:03.065397 10371 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:47:03.065410 10371 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:47:03.065423 10371 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:47:03.065478 10371 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:47:03.066948 10371 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:45851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056405724609-6305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056405724609-6305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.40.126:37985
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab
--principal=kudu/127.6.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.126:37985
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10371
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056405724609-6305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:47:03.067209 10371 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:47:03.067446 10371 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:47:03.072496 10383 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:03.072569 10379 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:03.072688 10371 server_base.cc:1034] running on GCE node
W20260806 22:47:03.072788 10381 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:03.073119 10371 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:47:03.073475 10371 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:47:03.074659 10371 hybrid_clock.cc:648] HybridClock initialized: now 1786056423074657 us; error 38 us; skew 500 ppm
Aug 06 22:47:03 dist-test-slave-zh2d krb5kdc[10355](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056423, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:47:03.077289 10371 init.cc:377] Logged in from keytab as kudu/127.6.40.126@KRBTEST.COM (short username kudu)
I20260806 22:47:03.077683 10371 webserver.cc:460] Webserver started at http://127.6.40.126:37747/ using document root <none> and password file <none>
I20260806 22:47:03.077865 10371 fs_manager.cc:359] Metadata directory not provided
I20260806 22:47:03.077904 10371 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:47:03.078011 10371 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:47:03.079043 10371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056405724609-6305-0/minicluster-data/master-0/data/instance:
uuid: "72754356e4624ffbb4b06d0765cce2d2"
format_stamp: "Formatted at 2026-08-06 22:47:03 on dist-test-slave-zh2d"
server_key: "cf36e36014e5f3061e18bb67356fe3da"
server_key_iv: "fbccc35b7e380c3993fb3ec21015cc8a"
server_key_version: "encryptionkey@0"
I20260806 22:47:03.079355 10371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056405724609-6305-0/minicluster-data/master-0/wal/instance:
uuid: "72754356e4624ffbb4b06d0765cce2d2"
format_stamp: "Formatted at 2026-08-06 22:47:03 on dist-test-slave-zh2d"
server_key: "cf36e36014e5f3061e18bb67356fe3da"
server_key_iv: "fbccc35b7e380c3993fb3ec21015cc8a"
server_key_version: "encryptionkey@0"
I20260806 22:47:03.080623 10371 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 22:47:03.081377 10399 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:03.081581 10371 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:03.081681 10371 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056405724609-6305-0/minicluster-data/master-0/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056405724609-6305-0/minicluster-data/master-0/wal
uuid: "72754356e4624ffbb4b06d0765cce2d2"
format_stamp: "Formatted at 2026-08-06 22:47:03 on dist-test-slave-zh2d"
server_key: "cf36e36014e5f3061e18bb67356fe3da"
server_key_iv: "fbccc35b7e380c3993fb3ec21015cc8a"
server_key_version: "encryptionkey@0"
I20260806 22:47:03.081771 10371 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056405724609-6305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056405724609-6305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056405724609-6305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:47:03.099398 10371 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:47:03.101377 10371 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:47:03.101541 10371 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:47:03.105957 10371 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.126:37985
I20260806 22:47:03.106046 10468 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.126:37985 every 8 connection(s)
I20260806 22:47:03.106410 10371 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
I20260806 22:47:03.107365 10469 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:47:03.109844 10469 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72754356e4624ffbb4b06d0765cce2d2: Bootstrap starting.
I20260806 22:47:03.110409 10469 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 72754356e4624ffbb4b06d0765cce2d2: Neither blocks nor log segments found. Creating new log.
I20260806 22:47:03.110687 10469 log.cc:824] T 00000000000000000000000000000000 P 72754356e4624ffbb4b06d0765cce2d2: Log is configured to *not* fsync() on all Append() calls
I20260806 22:47:03.111356 10469 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72754356e4624ffbb4b06d0765cce2d2: No bootstrap required, opened a new log
I20260806 22:47:03.113062 10469 raft_consensus.cc:348] T 00000000000000000000000000000000 P 72754356e4624ffbb4b06d0765cce2d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72754356e4624ffbb4b06d0765cce2d2" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 37985 } }
I20260806 22:47:03.113204 10469 raft_consensus.cc:374] T 00000000000000000000000000000000 P 72754356e4624ffbb4b06d0765cce2d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:47:03.113227 10469 raft_consensus.cc:729] T 00000000000000000000000000000000 P 72754356e4624ffbb4b06d0765cce2d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72754356e4624ffbb4b06d0765cce2d2, State: Initialized, Role: FOLLOWER
I20260806 22:47:03.113368 10469 consensus_queue.cc:260] T 00000000000000000000000000000000 P 72754356e4624ffbb4b06d0765cce2d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72754356e4624ffbb4b06d0765cce2d2" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 37985 } }
I20260806 22:47:03.113447 10469 raft_consensus.cc:388] T 00000000000000000000000000000000 P 72754356e4624ffbb4b06d0765cce2d2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:47:03.113478 10469 raft_consensus.cc:482] T 00000000000000000000000000000000 P 72754356e4624ffbb4b06d0765cce2d2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:47:03.113512 10469 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 72754356e4624ffbb4b06d0765cce2d2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:47:03.114274 10469 raft_consensus.cc:504] T 00000000000000000000000000000000 P 72754356e4624ffbb4b06d0765cce2d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72754356e4624ffbb4b06d0765cce2d2" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 37985 } }
I20260806 22:47:03.114359 10469 leader_election.cc:302] T 00000000000000000000000000000000 P 72754356e4624ffbb4b06d0765cce2d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 72754356e4624ffbb4b06d0765cce2d2; no voters:
I20260806 22:47:03.114495 10469 leader_election.cc:288] T 00000000000000000000000000000000 P 72754356e4624ffbb4b06d0765cce2d2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:47:03.114567 10473 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 72754356e4624ffbb4b06d0765cce2d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:47:03.114781 10473 raft_consensus.cc:686] T 00000000000000000000000000000000 P 72754356e4624ffbb4b06d0765cce2d2 [term 1 LEADER]: Becoming Leader. State: Replica: 72754356e4624ffbb4b06d0765cce2d2, State: Running, Role: LEADER
I20260806 22:47:03.114769 10469 sys_catalog.cc:564] T 00000000000000000000000000000000 P 72754356e4624ffbb4b06d0765cce2d2 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:47:03.114885 10473 consensus_queue.cc:237] T 00000000000000000000000000000000 P 72754356e4624ffbb4b06d0765cce2d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72754356e4624ffbb4b06d0765cce2d2" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 37985 } }
I20260806 22:47:03.115275 10476 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72754356e4624ffbb4b06d0765cce2d2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 72754356e4624ffbb4b06d0765cce2d2. Latest consensus state: current_term: 1 leader_uuid: "72754356e4624ffbb4b06d0765cce2d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72754356e4624ffbb4b06d0765cce2d2" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 37985 } } }
I20260806 22:47:03.115298 10474 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72754356e4624ffbb4b06d0765cce2d2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "72754356e4624ffbb4b06d0765cce2d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72754356e4624ffbb4b06d0765cce2d2" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 37985 } } }
I20260806 22:47:03.115402 10476 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72754356e4624ffbb4b06d0765cce2d2 [sys.catalog]: This master's current role is: LEADER
I20260806 22:47:03.115445 10474 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72754356e4624ffbb4b06d0765cce2d2 [sys.catalog]: This master's current role is: LEADER
I20260806 22:47:03.115859 10492 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:47:03.116053 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 10371
I20260806 22:47:03.116189 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056405724609-6305-0/minicluster-data/master-0/wal/instance
I20260806 22:47:03.116425 6305 external_mini_cluster.cc:1428] Setting key e51cc94a3ecfd92c3432914d1f45c9f0
I20260806 22:47:03.116544 10492 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:47:03.118229 10492 catalog_manager.cc:1347] Generated new cluster ID: 26d4c6f693794f1daf19ee44729c1682
I20260806 22:47:03.118285 10492 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 22:47:03 dist-test-slave-zh2d krb5kdc[10355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056422, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
I20260806 22:47:03.126940 10492 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:47:03.127568 10492 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:47:03.129069 10506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:03.116887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53902 (local address 127.6.40.126:37985)
0806 22:47:03.117364 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:03.117374 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:03.117410 (+ 36us) server_negotiation.cc:407] Connection header received
0806 22:47:03.117531 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:03.117538 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:03.117597 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:03.117739 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:03.118743 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:03.120195 (+ 1452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:03.121245 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:03.121904 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:03.122030 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:03.124946 (+ 2916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:03.124950 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:03.124951 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:03.124963 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:03.127262 (+ 2299us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:03.127691 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:03.127694 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:03.127695 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:03.127806 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:03.128053 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:03.128054 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:03.128054 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:03.128359 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:03.128544 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:03.128627 (+ 83us) server_negotiation.cc:299] Negotiation successful
0806 22:47:03.128784 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":423,"thread_start_us":99,"threads_started":1}
I20260806 22:47:03.130242 6305 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:47:03.133970 10492 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 72754356e4624ffbb4b06d0765cce2d2: Generated new TSK 0
I20260806 22:47:03.134198 10492 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:47:03.146680 10506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:03.142690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53906 (local address 127.6.40.126:37985)
0806 22:47:03.142758 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:03.142761 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:03.142903 (+ 142us) server_negotiation.cc:407] Connection header received
0806 22:47:03.142965 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:03.142966 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:03.142975 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:03.143032 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:03.143504 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:03.144047 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:03.145172 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:03.145396 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:03.145451 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:03.146171 (+ 720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:03.146172 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:03.146172 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:47:03.146367 (+ 195us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:03.146385 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:47:03.146391 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:03.146437 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:03.146544 (+ 107us) server_negotiation.cc:299] Negotiation successful
0806 22:47:03.146571 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 22:47:03.149513 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 10371
2026-08-06T22:47:03Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1553 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:47:03 dist-test-slave-zh2d krb5kdc[10523](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:47:03 dist-test-slave-zh2d krb5kdc[10523](info): set up 2 sockets
Aug 06 22:47:03 dist-test-slave-zh2d krb5kdc[10523](info): commencing operation
krb5kdc: starting...
W20260806 22:47:04.373817 6305 mini_kdc.cc:121] Time spent starting KDC: real 1.192s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/krb5kdc/test-user.keytab.
Aug 06 22:47:04 dist-test-slave-zh2d krb5kdc[10523](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056424, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:47:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:47:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.40.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.40.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.40.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab.
Entry for principal kudu/127.6.40.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab.
I20260806 22:47:04.505617 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.126:37273
--webserver_interface=127.6.40.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab
--principal=kudu/127.6.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:45911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.40.126:37273
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:47:04.588541 10542 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:47:04.588795 10542 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:47:04.588833 10542 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:47:04.589983 10542 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:47:04.590063 10542 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:47:04.590088 10542 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:47:04.590108 10542 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:47:04.590139 10542 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:47:04.591619 10542 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:45911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.40.126:37273
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab
--principal=kudu/127.6.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.126:37273
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10542
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:47:04.591900 10542 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:47:04.592139 10542 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 22:47:04.597378 10542 server_base.cc:1034] running on GCE node
W20260806 22:47:04.597316 10550 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:04.597339 10555 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:04.597299 10551 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:04.597932 10542 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:47:04.598271 10542 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:47:04.599472 10542 hybrid_clock.cc:648] HybridClock initialized: now 1786056424599451 us; error 30 us; skew 500 ppm
Aug 06 22:47:04 dist-test-slave-zh2d krb5kdc[10523](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056424, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:47:04.602103 10542 init.cc:377] Logged in from keytab as kudu/127.6.40.126@KRBTEST.COM (short username kudu)
I20260806 22:47:04.602505 10542 webserver.cc:460] Webserver started at http://127.6.40.126:41677/ using document root <none> and password file <none>
I20260806 22:47:04.602703 10542 fs_manager.cc:359] Metadata directory not provided
I20260806 22:47:04.602744 10542 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:47:04.602840 10542 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:47:04.603881 10542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/minicluster-data/master-0/data/instance:
uuid: "68353a86fe7c4e08b3bfa67fdc866b69"
format_stamp: "Formatted at 2026-08-06 22:47:04 on dist-test-slave-zh2d"
server_key: "ac922dd6a744874d3872af3d356e5202"
server_key_iv: "5cdbc941b2484c6c3860fa16a7e9d87e"
server_key_version: "encryptionkey@0"
I20260806 22:47:04.604209 10542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/minicluster-data/master-0/wal/instance:
uuid: "68353a86fe7c4e08b3bfa67fdc866b69"
format_stamp: "Formatted at 2026-08-06 22:47:04 on dist-test-slave-zh2d"
server_key: "ac922dd6a744874d3872af3d356e5202"
server_key_iv: "5cdbc941b2484c6c3860fa16a7e9d87e"
server_key_version: "encryptionkey@0"
I20260806 22:47:04.605579 10542 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:47:04.606292 10566 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:04.606474 10542 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:47:04.606554 10542 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/minicluster-data/master-0/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/minicluster-data/master-0/wal
uuid: "68353a86fe7c4e08b3bfa67fdc866b69"
format_stamp: "Formatted at 2026-08-06 22:47:04 on dist-test-slave-zh2d"
server_key: "ac922dd6a744874d3872af3d356e5202"
server_key_iv: "5cdbc941b2484c6c3860fa16a7e9d87e"
server_key_version: "encryptionkey@0"
I20260806 22:47:04.606627 10542 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:47:04.618917 10542 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:47:04.621408 10542 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:47:04.621582 10542 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:47:04.625795 10542 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.126:37273
I20260806 22:47:04.625869 10646 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.126:37273 every 8 connection(s)
I20260806 22:47:04.626233 10542 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
I20260806 22:47:04.627377 10647 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:47:04.629778 10647 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68353a86fe7c4e08b3bfa67fdc866b69: Bootstrap starting.
I20260806 22:47:04.630373 10647 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 68353a86fe7c4e08b3bfa67fdc866b69: Neither blocks nor log segments found. Creating new log.
I20260806 22:47:04.630681 10647 log.cc:824] T 00000000000000000000000000000000 P 68353a86fe7c4e08b3bfa67fdc866b69: Log is configured to *not* fsync() on all Append() calls
I20260806 22:47:04.631295 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 10542
I20260806 22:47:04.631402 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/minicluster-data/master-0/wal/instance
I20260806 22:47:04.631536 10647 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68353a86fe7c4e08b3bfa67fdc866b69: No bootstrap required, opened a new log
I20260806 22:47:04.631604 6305 external_mini_cluster.cc:1428] Setting key 86b807fc8d6ead67125885171f447828
I20260806 22:47:04.634073 10647 raft_consensus.cc:348] T 00000000000000000000000000000000 P 68353a86fe7c4e08b3bfa67fdc866b69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68353a86fe7c4e08b3bfa67fdc866b69" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 37273 } }
I20260806 22:47:04.634298 10647 raft_consensus.cc:374] T 00000000000000000000000000000000 P 68353a86fe7c4e08b3bfa67fdc866b69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:47:04.634351 10647 raft_consensus.cc:729] T 00000000000000000000000000000000 P 68353a86fe7c4e08b3bfa67fdc866b69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68353a86fe7c4e08b3bfa67fdc866b69, State: Initialized, Role: FOLLOWER
I20260806 22:47:04.634490 10647 consensus_queue.cc:260] T 00000000000000000000000000000000 P 68353a86fe7c4e08b3bfa67fdc866b69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68353a86fe7c4e08b3bfa67fdc866b69" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 37273 } }
I20260806 22:47:04.634589 10647 raft_consensus.cc:388] T 00000000000000000000000000000000 P 68353a86fe7c4e08b3bfa67fdc866b69 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:47:04.634635 10647 raft_consensus.cc:482] T 00000000000000000000000000000000 P 68353a86fe7c4e08b3bfa67fdc866b69 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:47:04.634684 10647 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 68353a86fe7c4e08b3bfa67fdc866b69 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:47:04.635234 10647 raft_consensus.cc:504] T 00000000000000000000000000000000 P 68353a86fe7c4e08b3bfa67fdc866b69 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68353a86fe7c4e08b3bfa67fdc866b69" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 37273 } }
I20260806 22:47:04.635357 10647 leader_election.cc:302] T 00000000000000000000000000000000 P 68353a86fe7c4e08b3bfa67fdc866b69 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 68353a86fe7c4e08b3bfa67fdc866b69; no voters:
I20260806 22:47:04.635556 10647 leader_election.cc:288] T 00000000000000000000000000000000 P 68353a86fe7c4e08b3bfa67fdc866b69 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:47:04.635596 10653 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 68353a86fe7c4e08b3bfa67fdc866b69 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:47:04.635758 10653 raft_consensus.cc:686] T 00000000000000000000000000000000 P 68353a86fe7c4e08b3bfa67fdc866b69 [term 1 LEADER]: Becoming Leader. State: Replica: 68353a86fe7c4e08b3bfa67fdc866b69, State: Running, Role: LEADER
I20260806 22:47:04.635814 10647 sys_catalog.cc:564] T 00000000000000000000000000000000 P 68353a86fe7c4e08b3bfa67fdc866b69 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:47:04.636574 10653 consensus_queue.cc:237] T 00000000000000000000000000000000 P 68353a86fe7c4e08b3bfa67fdc866b69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68353a86fe7c4e08b3bfa67fdc866b69" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 37273 } }
I20260806 22:47:04.637377 10654 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68353a86fe7c4e08b3bfa67fdc866b69 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 68353a86fe7c4e08b3bfa67fdc866b69. Latest consensus state: current_term: 1 leader_uuid: "68353a86fe7c4e08b3bfa67fdc866b69" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68353a86fe7c4e08b3bfa67fdc866b69" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 37273 } } }
I20260806 22:47:04.637470 10654 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68353a86fe7c4e08b3bfa67fdc866b69 [sys.catalog]: This master's current role is: LEADER
I20260806 22:47:04.637744 10655 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68353a86fe7c4e08b3bfa67fdc866b69 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "68353a86fe7c4e08b3bfa67fdc866b69" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68353a86fe7c4e08b3bfa67fdc866b69" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 37273 } } }
I20260806 22:47:04.637921 10655 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68353a86fe7c4e08b3bfa67fdc866b69 [sys.catalog]: This master's current role is: LEADER
W20260806 22:47:04.638480 10670 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 68353a86fe7c4e08b3bfa67fdc866b69: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:47:04.638549 10670 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:47:04.638654 10675 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:47:04.638831 10675 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 06 22:47:04 dist-test-slave-zh2d krb5kdc[10523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056424, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
I20260806 22:47:04.640398 10675 catalog_manager.cc:1347] Generated new cluster ID: 998c89f97a9347d0b19199845efcc05f
I20260806 22:47:04.640461 10675 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:47:04.644183 10652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:04.632147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46890 (local address 127.6.40.126:37273)
0806 22:47:04.632357 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:04.632365 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:04.632391 (+ 26us) server_negotiation.cc:407] Connection header received
0806 22:47:04.632970 (+ 579us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:04.632977 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:04.633034 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:04.633159 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:04.633980 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:04.635439 (+ 1459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:04.636478 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:04.637213 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:04.637338 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:04.640377 (+ 3039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:04.640381 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:04.640381 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:04.640393 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:04.642504 (+ 2111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:04.642911 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:04.642914 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:04.642914 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:04.643033 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:04.643257 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:04.643258 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:04.643258 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:04.643565 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:04.643731 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:04.643807 (+ 76us) server_negotiation.cc:299] Negotiation successful
0806 22:47:04.643953 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":124,"threads_started":1}
I20260806 22:47:04.658041 10675 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:47:04.658766 10675 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:47:04.667028 10675 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 68353a86fe7c4e08b3bfa67fdc866b69: Generated new TSK 0
I20260806 22:47:04.667225 10675 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.40.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.40.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.40.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.65.keytab.
Entry for principal kudu/127.6.40.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.65.keytab.
I20260806 22:47:04.703856 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.65:0
--local_ip_for_outbound_sockets=127.6.40.65
--webserver_interface=127.6.40.65
--webserver_port=0
--tserver_master_addrs=127.6.40.126:37273
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.65.keytab
--principal=kudu/127.6.40.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:45911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:47:04.790133 10683 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:47:04.790345 10683 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:47:04.790369 10683 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:47:04.791525 10683 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:47:04.791567 10683 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:47:04.791595 10683 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.65
I20260806 22:47:04.793056 10683 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:45911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.65.keytab
--principal=kudu/127.6.40.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.40.65
--webserver_port=0
--tserver_master_addrs=127.6.40.126:37273
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10683
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.65
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:47:04.793254 10683 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:47:04.793471 10683 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:47:04.798724 10691 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:04.798748 10693 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:04.798743 10695 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:04.798760 10683 server_base.cc:1034] running on GCE node
I20260806 22:47:04.799196 10683 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:47:04.799466 10683 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:47:04.800630 10683 hybrid_clock.cc:648] HybridClock initialized: now 1786056424800616 us; error 36 us; skew 500 ppm
Aug 06 22:47:04 dist-test-slave-zh2d krb5kdc[10523](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056424, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:47:04.803341 10683 init.cc:377] Logged in from keytab as kudu/127.6.40.65@KRBTEST.COM (short username kudu)
I20260806 22:47:04.803757 10683 webserver.cc:460] Webserver started at http://127.6.40.65:37455/ using document root <none> and password file <none>
I20260806 22:47:04.803954 10683 fs_manager.cc:359] Metadata directory not provided
I20260806 22:47:04.803999 10683 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:47:04.804091 10683 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:47:04.805240 10683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-0/data/instance:
uuid: "62acb6338a824f6fbf6833f34e577b61"
format_stamp: "Formatted at 2026-08-06 22:47:04 on dist-test-slave-zh2d"
server_key: "0ed784e8fbdfaa1997a0823d3cd13980"
server_key_iv: "0140c4582933171663683112218a56e4"
server_key_version: "encryptionkey@0"
I20260806 22:47:04.805568 10683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-0/wal/instance:
uuid: "62acb6338a824f6fbf6833f34e577b61"
format_stamp: "Formatted at 2026-08-06 22:47:04 on dist-test-slave-zh2d"
server_key: "0ed784e8fbdfaa1997a0823d3cd13980"
server_key_iv: "0140c4582933171663683112218a56e4"
server_key_version: "encryptionkey@0"
I20260806 22:47:04.806896 10683 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:47:04.807636 10704 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:04.807819 10683 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:47:04.807932 10683 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-0/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-0/wal
uuid: "62acb6338a824f6fbf6833f34e577b61"
format_stamp: "Formatted at 2026-08-06 22:47:04 on dist-test-slave-zh2d"
server_key: "0ed784e8fbdfaa1997a0823d3cd13980"
server_key_iv: "0140c4582933171663683112218a56e4"
server_key_version: "encryptionkey@0"
I20260806 22:47:04.808017 10683 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:47:04.817911 10683 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:47:04.820418 10683 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:47:04.820600 10683 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:47:04.820885 10683 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:47:04.821240 10683 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:47:04.821296 10683 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:04.821338 10683 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:47:04.821365 10683 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:04.827629 10683 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.65:38943
I20260806 22:47:04.827682 10868 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.65:38943 every 8 connection(s)
I20260806 22:47:04.828011 10683 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
I20260806 22:47:04.829512 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 10683
I20260806 22:47:04.829635 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-0/wal/instance
I20260806 22:47:04.829887 6305 external_mini_cluster.cc:1428] Setting key 24fdaec2d1f58033bd8aa81716fb13aa
Aug 06 22:47:04 dist-test-slave-zh2d krb5kdc[10523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056424, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.65@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
I20260806 22:47:04.838943 10652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:04.828775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.65:40059 (local address 127.6.40.126:37273)
0806 22:47:04.828890 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:04.828892 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:04.829092 (+ 200us) server_negotiation.cc:407] Connection header received
0806 22:47:04.829679 (+ 587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:04.829681 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:04.829695 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:04.829785 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:04.830984 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:04.831746 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:04.832828 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:04.833088 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:04.833184 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:04.835618 (+ 2434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:04.835620 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:04.835620 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:04.835628 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:04.837474 (+ 1846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:04.837900 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:04.837902 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:04.837902 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:04.837972 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:04.838214 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:04.838215 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:04.838215 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:04.838375 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:04.838458 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:04.838719 (+ 261us) server_negotiation.cc:299] Negotiation successful
0806 22:47:04.838818 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 22:47:04.839011 10871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:04.828724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:37273 (local address 127.6.40.65:40059)
0806 22:47:04.828917 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:04.828961 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:04.829515 (+ 554us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:04.829861 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:04.829865 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:04.830621 (+ 756us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:04.830870 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:04.830875 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:04.831847 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:04.831848 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:04.832732 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:04.832734 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:04.833201 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:04.833202 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:04.833335 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:04.833919 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:04.833928 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:04.835502 (+ 1574us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:47:04.837578 (+ 2076us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:04.837585 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:04.837588 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:04.837818 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:04.838044 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:04.838044 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:04.838045 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:04.838146 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:04.838449 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:47:04.838454 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:47:04.838581 (+ 127us) client_negotiation.cc:769] Sending connection context
0806 22:47:04.838667 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 22:47:04.838795 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":112,"threads_started":1}
I20260806 22:47:04.839605 10869 heartbeater.cc:347] Connected to a master server at 127.6.40.126:37273
I20260806 22:47:04.839720 10869 heartbeater.cc:464] Registering TS with master...
I20260806 22:47:04.839936 10869 heartbeater.cc:511] Master 127.6.40.126:37273 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.6.40.66@KRBTEST.COM; defaulting to no policy
I20260806 22:47:04.840559 10593 ts_manager.cc:194] Registered new tserver with Master: 62acb6338a824f6fbf6833f34e577b61 (127.6.40.65:38943)
I20260806 22:47:04.841364 10593 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.40.65@KRBTEST.COM'} at 127.6.40.65:40059
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.40.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.40.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.66.keytab.
Entry for principal kudu/127.6.40.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.66.keytab.
I20260806 22:47:04.859786 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.66:0
--local_ip_for_outbound_sockets=127.6.40.66
--webserver_interface=127.6.40.66
--webserver_port=0
--tserver_master_addrs=127.6.40.126:37273
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.66.keytab
--principal=kudu/127.6.40.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:45911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:47:04.944819 10877 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:47:04.945138 10877 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:47:04.945178 10877 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:47:04.946348 10877 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:47:04.946419 10877 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:47:04.946460 10877 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.66
I20260806 22:47:04.947934 10877 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:45911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.66.keytab
--principal=kudu/127.6.40.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.40.66
--webserver_port=0
--tserver_master_addrs=127.6.40.126:37273
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10877
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.66
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:47:04.948217 10877 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:47:04.948457 10877 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 22:47:04.953848 10877 server_base.cc:1034] running on GCE node
W20260806 22:47:04.953778 10886 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:04.953804 10888 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:04.953790 10890 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:04.954479 10877 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:47:04.954767 10877 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:47:04.955928 10877 hybrid_clock.cc:648] HybridClock initialized: now 1786056424955913 us; error 33 us; skew 500 ppm
Aug 06 22:47:04 dist-test-slave-zh2d krb5kdc[10523](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056424, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:47:04.958508 10877 init.cc:377] Logged in from keytab as kudu/127.6.40.66@KRBTEST.COM (short username kudu)
I20260806 22:47:04.958982 10877 webserver.cc:460] Webserver started at http://127.6.40.66:41107/ using document root <none> and password file <none>
I20260806 22:47:04.959249 10877 fs_manager.cc:359] Metadata directory not provided
I20260806 22:47:04.959319 10877 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:47:04.959465 10877 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:47:04.960530 10877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-1/data/instance:
uuid: "674118afba2547498cb8bb3caa740b2d"
format_stamp: "Formatted at 2026-08-06 22:47:04 on dist-test-slave-zh2d"
server_key: "0a01a40d51f89426b6c19ae2f10b619d"
server_key_iv: "4b1424f29aa243bea8ce8d4c84d0ce46"
server_key_version: "encryptionkey@0"
I20260806 22:47:04.960984 10877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-1/wal/instance:
uuid: "674118afba2547498cb8bb3caa740b2d"
format_stamp: "Formatted at 2026-08-06 22:47:04 on dist-test-slave-zh2d"
server_key: "0a01a40d51f89426b6c19ae2f10b619d"
server_key_iv: "4b1424f29aa243bea8ce8d4c84d0ce46"
server_key_version: "encryptionkey@0"
I20260806 22:47:04.962376 10877 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:47:04.963132 10903 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:04.963313 10877 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:47:04.963416 10877 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-1/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-1/wal
uuid: "674118afba2547498cb8bb3caa740b2d"
format_stamp: "Formatted at 2026-08-06 22:47:04 on dist-test-slave-zh2d"
server_key: "0a01a40d51f89426b6c19ae2f10b619d"
server_key_iv: "4b1424f29aa243bea8ce8d4c84d0ce46"
server_key_version: "encryptionkey@0"
I20260806 22:47:04.963500 10877 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:47:04.972358 10877 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:47:04.974589 10877 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:47:04.974762 10877 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:47:04.975046 10877 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:47:04.975379 10877 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:47:04.975435 10877 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:04.975479 10877 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:47:04.975507 10877 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:04.982285 10877 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.66:46181
I20260806 22:47:04.982327 11051 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.66:46181 every 8 connection(s)
I20260806 22:47:04.982676 10877 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
I20260806 22:47:04.985266 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 10877
I20260806 22:47:04.985352 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-1/wal/instance
I20260806 22:47:04.985558 6305 external_mini_cluster.cc:1428] Setting key 202b8e277bd2be0c9cebb0c8db214bb7
Aug 06 22:47:04 dist-test-slave-zh2d krb5kdc[10523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056424, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.66@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
I20260806 22:47:04.993432 10652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:04.983461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.66:53973 (local address 127.6.40.126:37273)
0806 22:47:04.983533 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:04.983535 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:04.983764 (+ 229us) server_negotiation.cc:407] Connection header received
0806 22:47:04.984282 (+ 518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:04.984283 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:04.984294 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:04.984350 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:04.985460 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:04.986091 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:04.987134 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:04.987339 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:04.987437 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:04.990157 (+ 2720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:04.990159 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:04.990159 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:04.990169 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:04.991879 (+ 1710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:04.992312 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:04.992313 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:04.992313 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:04.992383 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:04.992639 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:04.992640 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:04.992641 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:04.992811 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:04.992938 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:04.993227 (+ 289us) server_negotiation.cc:299] Negotiation successful
0806 22:47:04.993319 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 22:47:04.993501 11055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:04.983427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:37273 (local address 127.6.40.66:53973)
0806 22:47:04.983626 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:04.983671 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:04.984198 (+ 527us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:04.984413 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:04.984417 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:04.985099 (+ 682us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:04.985306 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:04.985329 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:04.986200 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:04.986202 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:04.987038 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:04.987041 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:04.987442 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:04.987443 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:04.987575 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:04.988168 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:04.988180 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:04.990055 (+ 1875us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:47:04.991979 (+ 1924us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:04.991987 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:04.991989 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:04.992232 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:04.992457 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:04.992458 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:04.992458 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:04.992571 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:04.992936 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:47:04.992942 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:47:04.993081 (+ 139us) client_negotiation.cc:769] Sending connection context
0806 22:47:04.993169 (+ 88us) client_negotiation.cc:240] Negotiation successful
0806 22:47:04.993302 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":115,"threads_started":1}
I20260806 22:47:04.994091 11052 heartbeater.cc:347] Connected to a master server at 127.6.40.126:37273
I20260806 22:47:04.994208 11052 heartbeater.cc:464] Registering TS with master...
I20260806 22:47:04.994423 11052 heartbeater.cc:511] Master 127.6.40.126:37273 requested a full tablet report, sending...
I20260806 22:47:04.994899 10593 ts_manager.cc:194] Registered new tserver with Master: 674118afba2547498cb8bb3caa740b2d (127.6.40.66:46181)
I20260806 22:47:04.995460 10593 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.40.66@KRBTEST.COM'} at 127.6.40.66:53973
WARNING: no policy specified for kudu/127.6.40.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.40.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.40.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.67.keytab.
Entry for principal kudu/127.6.40.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.67.keytab.
I20260806 22:47:05.016490 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.67:0
--local_ip_for_outbound_sockets=127.6.40.67
--webserver_interface=127.6.40.67
--webserver_port=0
--tserver_master_addrs=127.6.40.126:37273
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.67.keytab
--principal=kudu/127.6.40.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:45911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:47:05.104513 11060 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:47:05.104720 11060 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:47:05.104744 11060 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:47:05.106105 11060 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:47:05.106153 11060 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:47:05.106218 11060 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.67
I20260806 22:47:05.107649 11060 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:45911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.67.keytab
--principal=kudu/127.6.40.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.40.67
--webserver_port=0
--tserver_master_addrs=127.6.40.126:37273
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11060
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.67
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:47:05.107836 11060 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:47:05.108028 11060 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:47:05.114087 11066 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:05.114099 11067 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:05.114104 11071 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:05.114589 11060 server_base.cc:1034] running on GCE node
I20260806 22:47:05.114789 11060 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:47:05.115011 11060 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:47:05.116170 11060 hybrid_clock.cc:648] HybridClock initialized: now 1786056425116158 us; error 33 us; skew 500 ppm
Aug 06 22:47:05 dist-test-slave-zh2d krb5kdc[10523](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056425, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:47:05.119082 11060 init.cc:377] Logged in from keytab as kudu/127.6.40.67@KRBTEST.COM (short username kudu)
I20260806 22:47:05.119531 11060 webserver.cc:460] Webserver started at http://127.6.40.67:32955/ using document root <none> and password file <none>
I20260806 22:47:05.119769 11060 fs_manager.cc:359] Metadata directory not provided
I20260806 22:47:05.119844 11060 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:47:05.119988 11060 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:47:05.121135 11060 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-2/data/instance:
uuid: "16df549bfeea4f12a6525dc4e1ca69e6"
format_stamp: "Formatted at 2026-08-06 22:47:05 on dist-test-slave-zh2d"
server_key: "994ddbffee90a062e6dee6e90fa9180b"
server_key_iv: "991e200d1ded40fd99055973106d065b"
server_key_version: "encryptionkey@0"
I20260806 22:47:05.121511 11060 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-2/wal/instance:
uuid: "16df549bfeea4f12a6525dc4e1ca69e6"
format_stamp: "Formatted at 2026-08-06 22:47:05 on dist-test-slave-zh2d"
server_key: "994ddbffee90a062e6dee6e90fa9180b"
server_key_iv: "991e200d1ded40fd99055973106d065b"
server_key_version: "encryptionkey@0"
I20260806 22:47:05.123050 11060 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:47:05.123824 11080 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:05.124006 11060 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:47:05.124109 11060 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-2/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-2/wal
uuid: "16df549bfeea4f12a6525dc4e1ca69e6"
format_stamp: "Formatted at 2026-08-06 22:47:05 on dist-test-slave-zh2d"
server_key: "994ddbffee90a062e6dee6e90fa9180b"
server_key_iv: "991e200d1ded40fd99055973106d065b"
server_key_version: "encryptionkey@0"
I20260806 22:47:05.124188 11060 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:47:05.134845 11060 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:47:05.137243 11060 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:47:05.137396 11060 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:47:05.137638 11060 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:47:05.137957 11060 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:47:05.138017 11060 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:05.138077 11060 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:47:05.138099 11060 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:05.144338 11060 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.67:35467
I20260806 22:47:05.144369 11240 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.67:35467 every 8 connection(s)
I20260806 22:47:05.144728 11060 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
Aug 06 22:47:05 dist-test-slave-zh2d krb5kdc[10523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056425, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.67@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
I20260806 22:47:05.152973 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 11060
I20260806 22:47:05.153097 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056405724609-6305-0/minicluster-data/ts-2/wal/instance
I20260806 22:47:05.153301 6305 external_mini_cluster.cc:1428] Setting key b367f1d5c4ba8a48ccf4ccc325833221
I20260806 22:47:05.154568 10652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:05.145483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.67:38557 (local address 127.6.40.126:37273)
0806 22:47:05.145527 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:05.145530 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:05.145719 (+ 189us) server_negotiation.cc:407] Connection header received
0806 22:47:05.146300 (+ 581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:05.146302 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:05.146315 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:05.146376 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:05.147336 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:05.147865 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:05.149005 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:05.149226 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:05.149316 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:05.151508 (+ 2192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:05.151510 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:05.151510 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:05.151518 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:05.152970 (+ 1452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:05.153521 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:05.153522 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:05.153523 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:05.153580 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:05.153873 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:05.153874 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:05.153874 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:05.154045 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:05.154104 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:05.154419 (+ 315us) server_negotiation.cc:299] Negotiation successful
0806 22:47:05.154490 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 22:47:05.154752 11245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:05.145436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:37273 (local address 127.6.40.67:38557)
0806 22:47:05.145585 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:05.145630 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:05.146193 (+ 563us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:05.146445 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:05.146449 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:05.147041 (+ 592us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:05.147233 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:05.147237 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:05.147983 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:05.147984 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:05.148803 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:05.148805 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:05.149332 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:05.149334 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:05.149476 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:05.150022 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:05.150049 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:05.151391 (+ 1342us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:47:05.153083 (+ 1692us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:05.153092 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:05.153094 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:05.153419 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:05.153631 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:05.153632 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:05.153633 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:05.153775 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:05.154121 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:47:05.154128 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:47:05.154287 (+ 159us) client_negotiation.cc:769] Sending connection context
0806 22:47:05.154390 (+ 103us) client_negotiation.cc:240] Negotiation successful
0806 22:47:05.154521 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":73,"threads_started":1}
I20260806 22:47:05.155208 11242 heartbeater.cc:347] Connected to a master server at 127.6.40.126:37273
I20260806 22:47:05.155321 11242 heartbeater.cc:464] Registering TS with master...
I20260806 22:47:05.155529 11242 heartbeater.cc:511] Master 127.6.40.126:37273 requested a full tablet report, sending...
I20260806 22:47:05.155968 10593 ts_manager.cc:194] Registered new tserver with Master: 16df549bfeea4f12a6525dc4e1ca69e6 (127.6.40.67:35467)
I20260806 22:47:05.156425 10593 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.40.67@KRBTEST.COM'} at 127.6.40.67:38557
I20260806 22:47:05.165220 6305 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 22:47:05 dist-test-slave-zh2d krb5kdc[10523](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056425, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 06 22:47:05 dist-test-slave-zh2d krb5kdc[10523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056425, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
I20260806 22:47:05.190255 10652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:05.182804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46906 (local address 127.6.40.126:37273)
0806 22:47:05.182887 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:05.182890 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:05.182989 (+ 99us) server_negotiation.cc:407] Connection header received
0806 22:47:05.183113 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:05.183114 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:05.183133 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:05.183203 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:05.183825 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:05.184334 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:05.185232 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:05.185432 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:05.185532 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:05.187922 (+ 2390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:05.187924 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:05.187924 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:05.187931 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:05.189244 (+ 1313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:05.189563 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:05.189564 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:05.189564 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:05.189616 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:05.189781 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:05.189781 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:05.189781 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:05.189928 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:05.190016 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:05.190091 (+ 75us) server_negotiation.cc:299] Negotiation successful
0806 22:47:05.190154 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
W20260806 22:47:05.190546 10593 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:46906
I20260806 22:47:05.191012 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 10683
I20260806 22:47:05.210908 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 10877
I20260806 22:47:05.218847 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 11060
I20260806 22:47:05.229841 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 10542
2026-08-06T22:47:05Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2076 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:47:05 dist-test-slave-zh2d krb5kdc[11261](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:47:05 dist-test-slave-zh2d krb5kdc[11261](info): set up 2 sockets
Aug 06 22:47:05 dist-test-slave-zh2d krb5kdc[11261](info): commencing operation
krb5kdc: starting...
W20260806 22:47:06.466173 6305 mini_kdc.cc:121] Time spent starting KDC: real 1.208s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/krb5kdc/test-user.keytab.
Aug 06 22:47:06 dist-test-slave-zh2d krb5kdc[11261](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056426, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:47:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:47:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.40.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.40.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.40.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab.
Entry for principal kudu/127.6.40.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab.
I20260806 22:47:06.596182 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.126:37687
--webserver_interface=127.6.40.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab
--principal=kudu/127.6.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:35965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.40.126:37687
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:47:06.679278 11280 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:47:06.679533 11280 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:47:06.679569 11280 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:47:06.680709 11280 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:47:06.680781 11280 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:47:06.680805 11280 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:47:06.680828 11280 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:47:06.680882 11280 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:47:06.682461 11280 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:35965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.40.126:37687
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab
--principal=kudu/127.6.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.126:37687
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11280
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:47:06.682740 11280 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:47:06.682976 11280 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:47:06.688030 11288 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:06.688071 11290 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:06.688071 11287 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:06.688135 11280 server_base.cc:1034] running on GCE node
I20260806 22:47:06.688537 11280 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:47:06.688885 11280 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:47:06.690058 11280 hybrid_clock.cc:648] HybridClock initialized: now 1786056426690041 us; error 36 us; skew 500 ppm
Aug 06 22:47:06 dist-test-slave-zh2d krb5kdc[11261](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056426, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:47:06.695535 11280 init.cc:377] Logged in from keytab as kudu/127.6.40.126@KRBTEST.COM (short username kudu)
I20260806 22:47:06.696056 11280 webserver.cc:460] Webserver started at http://127.6.40.126:37627/ using document root <none> and password file <none>
I20260806 22:47:06.696290 11280 fs_manager.cc:359] Metadata directory not provided
I20260806 22:47:06.696338 11280 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:47:06.696473 11280 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:47:06.697670 11280 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/minicluster-data/master-0/data/instance:
uuid: "291e3b2cd2d74596a8bc0eb20e744535"
format_stamp: "Formatted at 2026-08-06 22:47:06 on dist-test-slave-zh2d"
server_key: "1607dc050cf0c38536803371b70ae080"
server_key_iv: "c1e0b3e9fffb2f200ffb3f49a867ab74"
server_key_version: "encryptionkey@0"
I20260806 22:47:06.698056 11280 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/minicluster-data/master-0/wal/instance:
uuid: "291e3b2cd2d74596a8bc0eb20e744535"
format_stamp: "Formatted at 2026-08-06 22:47:06 on dist-test-slave-zh2d"
server_key: "1607dc050cf0c38536803371b70ae080"
server_key_iv: "c1e0b3e9fffb2f200ffb3f49a867ab74"
server_key_version: "encryptionkey@0"
I20260806 22:47:06.699505 11280 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 22:47:06.700332 11301 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:06.700567 11280 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 22:47:06.700677 11280 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/minicluster-data/master-0/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/minicluster-data/master-0/wal
uuid: "291e3b2cd2d74596a8bc0eb20e744535"
format_stamp: "Formatted at 2026-08-06 22:47:06 on dist-test-slave-zh2d"
server_key: "1607dc050cf0c38536803371b70ae080"
server_key_iv: "c1e0b3e9fffb2f200ffb3f49a867ab74"
server_key_version: "encryptionkey@0"
I20260806 22:47:06.700757 11280 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:47:06.730005 11280 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:47:06.732925 11280 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:47:06.733109 11280 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:47:06.738739 11280 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.126:37687
I20260806 22:47:06.738786 11374 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.126:37687 every 8 connection(s)
I20260806 22:47:06.739530 11280 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
I20260806 22:47:06.740411 11377 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:47:06.743415 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 11280
I20260806 22:47:06.743536 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/minicluster-data/master-0/wal/instance
I20260806 22:47:06.743767 6305 external_mini_cluster.cc:1428] Setting key 3c2df62f26dae9af1caa195b9d20caaa
I20260806 22:47:06.744030 11377 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 291e3b2cd2d74596a8bc0eb20e744535: Bootstrap starting.
I20260806 22:47:06.744812 11377 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 291e3b2cd2d74596a8bc0eb20e744535: Neither blocks nor log segments found. Creating new log.
I20260806 22:47:06.745239 11377 log.cc:824] T 00000000000000000000000000000000 P 291e3b2cd2d74596a8bc0eb20e744535: Log is configured to *not* fsync() on all Append() calls
I20260806 22:47:06.746096 11377 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 291e3b2cd2d74596a8bc0eb20e744535: No bootstrap required, opened a new log
I20260806 22:47:06.749046 11377 raft_consensus.cc:348] T 00000000000000000000000000000000 P 291e3b2cd2d74596a8bc0eb20e744535 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "291e3b2cd2d74596a8bc0eb20e744535" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 37687 } }
I20260806 22:47:06.749214 11377 raft_consensus.cc:374] T 00000000000000000000000000000000 P 291e3b2cd2d74596a8bc0eb20e744535 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:47:06.749257 11377 raft_consensus.cc:729] T 00000000000000000000000000000000 P 291e3b2cd2d74596a8bc0eb20e744535 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 291e3b2cd2d74596a8bc0eb20e744535, State: Initialized, Role: FOLLOWER
I20260806 22:47:06.749410 11377 consensus_queue.cc:260] T 00000000000000000000000000000000 P 291e3b2cd2d74596a8bc0eb20e744535 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "291e3b2cd2d74596a8bc0eb20e744535" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 37687 } }
I20260806 22:47:06.749505 11377 raft_consensus.cc:388] T 00000000000000000000000000000000 P 291e3b2cd2d74596a8bc0eb20e744535 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:47:06.749545 11377 raft_consensus.cc:482] T 00000000000000000000000000000000 P 291e3b2cd2d74596a8bc0eb20e744535 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:47:06.749588 11377 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 291e3b2cd2d74596a8bc0eb20e744535 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:47:06.750357 11377 raft_consensus.cc:504] T 00000000000000000000000000000000 P 291e3b2cd2d74596a8bc0eb20e744535 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "291e3b2cd2d74596a8bc0eb20e744535" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 37687 } }
I20260806 22:47:06.750484 11377 leader_election.cc:302] T 00000000000000000000000000000000 P 291e3b2cd2d74596a8bc0eb20e744535 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 291e3b2cd2d74596a8bc0eb20e744535; no voters:
I20260806 22:47:06.750658 11377 leader_election.cc:288] T 00000000000000000000000000000000 P 291e3b2cd2d74596a8bc0eb20e744535 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:47:06.750845 11391 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 291e3b2cd2d74596a8bc0eb20e744535 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:47:06.750946 11377 sys_catalog.cc:564] T 00000000000000000000000000000000 P 291e3b2cd2d74596a8bc0eb20e744535 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:47:06.751075 11391 raft_consensus.cc:686] T 00000000000000000000000000000000 P 291e3b2cd2d74596a8bc0eb20e744535 [term 1 LEADER]: Becoming Leader. State: Replica: 291e3b2cd2d74596a8bc0eb20e744535, State: Running, Role: LEADER
I20260806 22:47:06.751220 11391 consensus_queue.cc:237] T 00000000000000000000000000000000 P 291e3b2cd2d74596a8bc0eb20e744535 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "291e3b2cd2d74596a8bc0eb20e744535" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 37687 } }
I20260806 22:47:06.751636 11391 sys_catalog.cc:455] T 00000000000000000000000000000000 P 291e3b2cd2d74596a8bc0eb20e744535 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 291e3b2cd2d74596a8bc0eb20e744535. Latest consensus state: current_term: 1 leader_uuid: "291e3b2cd2d74596a8bc0eb20e744535" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "291e3b2cd2d74596a8bc0eb20e744535" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 37687 } } }
I20260806 22:47:06.751801 11391 sys_catalog.cc:458] T 00000000000000000000000000000000 P 291e3b2cd2d74596a8bc0eb20e744535 [sys.catalog]: This master's current role is: LEADER
I20260806 22:47:06.752018 11390 sys_catalog.cc:455] T 00000000000000000000000000000000 P 291e3b2cd2d74596a8bc0eb20e744535 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "291e3b2cd2d74596a8bc0eb20e744535" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "291e3b2cd2d74596a8bc0eb20e744535" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 37687 } } }
I20260806 22:47:06.752111 11390 sys_catalog.cc:458] T 00000000000000000000000000000000 P 291e3b2cd2d74596a8bc0eb20e744535 [sys.catalog]: This master's current role is: LEADER
I20260806 22:47:06.753250 11406 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:47:06.753823 11406 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:47:06.755345 11406 catalog_manager.cc:1347] Generated new cluster ID: dbcdbed80abd4a049cbbe46bf19b9963
I20260806 22:47:06.755401 11406 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 22:47:06 dist-test-slave-zh2d krb5kdc[11261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056426, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
I20260806 22:47:06.766727 11385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:06.744547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50450 (local address 127.6.40.126:37687)
0806 22:47:06.744750 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:06.744761 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:06.749079 (+ 4318us) server_negotiation.cc:407] Connection header received
0806 22:47:06.749249 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:06.749258 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:06.749324 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:06.749489 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:06.750468 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:06.752068 (+ 1600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:06.753258 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:06.753993 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:06.754117 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:06.762394 (+ 8277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:06.762400 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:06.762401 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:06.762417 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:06.765052 (+ 2635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:06.765432 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:06.765436 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:06.765436 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:06.765537 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:06.765767 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:06.765768 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:06.765768 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:06.766029 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:06.766220 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:06.766298 (+ 78us) server_negotiation.cc:299] Negotiation successful
0806 22:47:06.766468 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":88,"threads_started":1}
I20260806 22:47:06.774955 11406 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:47:06.775710 11406 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:47:06.781877 11406 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 291e3b2cd2d74596a8bc0eb20e744535: Generated new TSK 0
I20260806 22:47:06.782083 11406 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.40.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.40.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.40.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.65.keytab.
Entry for principal kudu/127.6.40.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.65.keytab.
I20260806 22:47:06.819540 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.65:0
--local_ip_for_outbound_sockets=127.6.40.65
--webserver_interface=127.6.40.65
--webserver_port=0
--tserver_master_addrs=127.6.40.126:37687
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.65.keytab
--principal=kudu/127.6.40.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:35965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:47:06.912190 11421 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:47:06.912410 11421 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:47:06.912436 11421 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:47:06.913676 11421 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:47:06.913726 11421 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:47:06.913754 11421 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.65
I20260806 22:47:06.915385 11421 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:35965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.65.keytab
--principal=kudu/127.6.40.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.40.65
--webserver_port=0
--tserver_master_addrs=127.6.40.126:37687
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11421
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.65
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:47:06.915616 11421 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:47:06.915809 11421 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:47:06.921168 11428 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:06.921159 11430 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:06.921407 11421 server_base.cc:1034] running on GCE node
W20260806 22:47:06.921159 11427 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:06.921761 11421 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:47:06.922029 11421 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:47:06.923188 11421 hybrid_clock.cc:648] HybridClock initialized: now 1786056426923174 us; error 33 us; skew 500 ppm
Aug 06 22:47:06 dist-test-slave-zh2d krb5kdc[11261](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056426, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:47:06.925918 11421 init.cc:377] Logged in from keytab as kudu/127.6.40.65@KRBTEST.COM (short username kudu)
I20260806 22:47:06.926369 11421 webserver.cc:460] Webserver started at http://127.6.40.65:39001/ using document root <none> and password file <none>
I20260806 22:47:06.926623 11421 fs_manager.cc:359] Metadata directory not provided
I20260806 22:47:06.926690 11421 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:47:06.926831 11421 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:47:06.927879 11421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/minicluster-data/ts-0/data/instance:
uuid: "89c4e4ef09f14c9fbe37d6a4bb54197d"
format_stamp: "Formatted at 2026-08-06 22:47:06 on dist-test-slave-zh2d"
server_key: "d0cf9b61d5681cc7a5aa1b74fed0b89f"
server_key_iv: "a0ccb4c839f955dee00294341deba1ba"
server_key_version: "encryptionkey@0"
I20260806 22:47:06.928253 11421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/minicluster-data/ts-0/wal/instance:
uuid: "89c4e4ef09f14c9fbe37d6a4bb54197d"
format_stamp: "Formatted at 2026-08-06 22:47:06 on dist-test-slave-zh2d"
server_key: "d0cf9b61d5681cc7a5aa1b74fed0b89f"
server_key_iv: "a0ccb4c839f955dee00294341deba1ba"
server_key_version: "encryptionkey@0"
I20260806 22:47:06.929719 11421 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:47:06.930513 11441 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:06.930693 11421 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 22:47:06.930800 11421 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/minicluster-data/ts-0/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/minicluster-data/ts-0/wal
uuid: "89c4e4ef09f14c9fbe37d6a4bb54197d"
format_stamp: "Formatted at 2026-08-06 22:47:06 on dist-test-slave-zh2d"
server_key: "d0cf9b61d5681cc7a5aa1b74fed0b89f"
server_key_iv: "a0ccb4c839f955dee00294341deba1ba"
server_key_version: "encryptionkey@0"
I20260806 22:47:06.930876 11421 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:47:06.940490 11421 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:47:06.942940 11421 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:47:06.943113 11421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:47:06.943358 11421 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:47:06.943671 11421 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:47:06.943725 11421 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:06.943766 11421 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:47:06.943790 11421 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:06.950075 11421 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.65:33567
I20260806 22:47:06.950191 11602 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.65:33567 every 8 connection(s)
I20260806 22:47:06.950476 11421 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
I20260806 22:47:06.955883 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 11421
I20260806 22:47:06.955986 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/minicluster-data/ts-0/wal/instance
I20260806 22:47:06.956187 6305 external_mini_cluster.cc:1428] Setting key fae5b14bff4236ed8f80315ed4fa92b5
Aug 06 22:47:06 dist-test-slave-zh2d krb5kdc[11261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056426, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.65@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
I20260806 22:47:06.961030 11385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:06.951300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.65:57267 (local address 127.6.40.126:37687)
0806 22:47:06.951353 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:06.951355 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:06.951614 (+ 259us) server_negotiation.cc:407] Connection header received
0806 22:47:06.952570 (+ 956us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:06.952571 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:06.952582 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:06.952637 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:06.953307 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:06.953826 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:06.954825 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:06.955041 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:06.955097 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:06.957617 (+ 2520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:06.957618 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:06.957619 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:06.957630 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:06.959395 (+ 1765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:06.959918 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:06.959919 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:06.959919 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:06.959975 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:06.960259 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:06.960260 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:06.960260 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:06.960358 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:06.960434 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:06.960766 (+ 332us) server_negotiation.cc:299] Negotiation successful
0806 22:47:06.960885 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 22:47:06.961103 11606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:06.951302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:37687 (local address 127.6.40.65:57267)
0806 22:47:06.951475 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:06.951512 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:06.952484 (+ 972us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:06.952690 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:06.952693 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:06.953008 (+ 315us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:06.953229 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:06.953233 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:06.953913 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:06.953914 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:06.954745 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:06.954747 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:06.955120 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:06.955121 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:06.955223 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:06.955776 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:06.955787 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:06.957498 (+ 1711us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:47:06.959497 (+ 1999us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:06.959506 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:06.959508 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:06.959836 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:06.960048 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:06.960049 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:06.960049 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:06.960194 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:06.960422 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:47:06.960428 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:47:06.960605 (+ 177us) client_negotiation.cc:769] Sending connection context
0806 22:47:06.960721 (+ 116us) client_negotiation.cc:240] Negotiation successful
0806 22:47:06.960909 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":104,"threads_started":1}
I20260806 22:47:06.961659 11603 heartbeater.cc:347] Connected to a master server at 127.6.40.126:37687
I20260806 22:47:06.961772 11603 heartbeater.cc:464] Registering TS with master...
I20260806 22:47:06.961951 11603 heartbeater.cc:511] Master 127.6.40.126:37687 requested a full tablet report, sending...
I20260806 22:47:06.962539 11325 ts_manager.cc:194] Registered new tserver with Master: 89c4e4ef09f14c9fbe37d6a4bb54197d (127.6.40.65:33567)
I20260806 22:47:06.963500 11325 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.40.65@KRBTEST.COM'} at 127.6.40.65:57267
WARNING: no policy specified for kudu/127.6.40.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.40.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.40.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.66.keytab.
Entry for principal kudu/127.6.40.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.66.keytab.
I20260806 22:47:06.984181 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.66:0
--local_ip_for_outbound_sockets=127.6.40.66
--webserver_interface=127.6.40.66
--webserver_port=0
--tserver_master_addrs=127.6.40.126:37687
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.66.keytab
--principal=kudu/127.6.40.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:35965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:47:07.068812 11610 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:47:07.069044 11610 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:47:07.069067 11610 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:47:07.070152 11610 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:47:07.070184 11610 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:47:07.070209 11610 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.66
I20260806 22:47:07.071626 11610 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:35965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.66.keytab
--principal=kudu/127.6.40.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.40.66
--webserver_port=0
--tserver_master_addrs=127.6.40.126:37687
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11610
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.66
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:47:07.071803 11610 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:47:07.071979 11610 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 22:47:07.077178 11610 server_base.cc:1034] running on GCE node
W20260806 22:47:07.077126 11616 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:07.077118 11617 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:07.077122 11620 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:07.077674 11610 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:47:07.077917 11610 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:47:07.079062 11610 hybrid_clock.cc:648] HybridClock initialized: now 1786056427079049 us; error 30 us; skew 500 ppm
Aug 06 22:47:07 dist-test-slave-zh2d krb5kdc[11261](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056427, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:47:07.081694 11610 init.cc:377] Logged in from keytab as kudu/127.6.40.66@KRBTEST.COM (short username kudu)
I20260806 22:47:07.082140 11610 webserver.cc:460] Webserver started at http://127.6.40.66:35795/ using document root <none> and password file <none>
I20260806 22:47:07.082374 11610 fs_manager.cc:359] Metadata directory not provided
I20260806 22:47:07.082445 11610 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:47:07.082613 11610 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:47:07.083709 11610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/minicluster-data/ts-1/data/instance:
uuid: "fd54f33dfe474e71ad88609fa0a42e13"
format_stamp: "Formatted at 2026-08-06 22:47:07 on dist-test-slave-zh2d"
server_key: "04b1941023e1aaf4015666b19957f652"
server_key_iv: "f1b55b05f01656ac73def28e1e3341de"
server_key_version: "encryptionkey@0"
I20260806 22:47:07.084093 11610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/minicluster-data/ts-1/wal/instance:
uuid: "fd54f33dfe474e71ad88609fa0a42e13"
format_stamp: "Formatted at 2026-08-06 22:47:07 on dist-test-slave-zh2d"
server_key: "04b1941023e1aaf4015666b19957f652"
server_key_iv: "f1b55b05f01656ac73def28e1e3341de"
server_key_version: "encryptionkey@0"
I20260806 22:47:07.085628 11610 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:47:07.086405 11629 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:07.086621 11610 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:47:07.086733 11610 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/minicluster-data/ts-1/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/minicluster-data/ts-1/wal
uuid: "fd54f33dfe474e71ad88609fa0a42e13"
format_stamp: "Formatted at 2026-08-06 22:47:07 on dist-test-slave-zh2d"
server_key: "04b1941023e1aaf4015666b19957f652"
server_key_iv: "f1b55b05f01656ac73def28e1e3341de"
server_key_version: "encryptionkey@0"
I20260806 22:47:07.086808 11610 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:47:07.093808 11610 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:47:07.095615 11610 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:47:07.095777 11610 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:47:07.096007 11610 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:47:07.096315 11610 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:47:07.096372 11610 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:07.096412 11610 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:47:07.096441 11610 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:07.102509 11610 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.66:45523
I20260806 22:47:07.102595 11805 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.66:45523 every 8 connection(s)
I20260806 22:47:07.102967 11610 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
I20260806 22:47:07.109349 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 11610
I20260806 22:47:07.109521 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/minicluster-data/ts-1/wal/instance
I20260806 22:47:07.109719 6305 external_mini_cluster.cc:1428] Setting key 2e9bbe3a09cb80de2b7c4c9bb37ddc78
Aug 06 22:47:07 dist-test-slave-zh2d krb5kdc[11261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056427, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.66@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
I20260806 22:47:07.113600 11385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:07.103780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.66:43131 (local address 127.6.40.126:37687)
0806 22:47:07.103852 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:07.103854 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:07.104083 (+ 229us) server_negotiation.cc:407] Connection header received
0806 22:47:07.105212 (+ 1129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:07.105213 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:07.105230 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:07.105350 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:07.106022 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:07.106583 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:07.107603 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:07.107990 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:07.108084 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:07.110624 (+ 2540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:07.110626 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:07.110626 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:07.110634 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:07.112081 (+ 1447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:07.112523 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:07.112524 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:07.112525 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:07.112591 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:07.112879 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:07.112880 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:07.112880 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:07.113002 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:07.113101 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:07.113388 (+ 287us) server_negotiation.cc:299] Negotiation successful
0806 22:47:07.113487 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 22:47:07.113756 11810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:07.103759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:37687 (local address 127.6.40.66:43131)
0806 22:47:07.103940 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:07.103983 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:07.105061 (+ 1078us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:07.105435 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:07.105440 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:07.105724 (+ 284us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:07.105916 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:07.105921 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:07.106673 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:07.106673 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:07.107499 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:07.107501 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:07.108088 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:07.108089 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:07.108189 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:07.108772 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:07.108780 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:07.110487 (+ 1707us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:47:07.112184 (+ 1697us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:07.112191 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:07.112194 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:07.112420 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:07.112668 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:07.112669 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:07.112670 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:07.112781 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:07.113104 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:47:07.113113 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:47:07.113250 (+ 137us) client_negotiation.cc:769] Sending connection context
0806 22:47:07.113353 (+ 103us) client_negotiation.cc:240] Negotiation successful
0806 22:47:07.113521 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":91,"threads_started":1}
I20260806 22:47:07.114202 11808 heartbeater.cc:347] Connected to a master server at 127.6.40.126:37687
I20260806 22:47:07.114320 11808 heartbeater.cc:464] Registering TS with master...
I20260806 22:47:07.114506 11808 heartbeater.cc:511] Master 127.6.40.126:37687 requested a full tablet report, sending...
I20260806 22:47:07.114918 11325 ts_manager.cc:194] Registered new tserver with Master: fd54f33dfe474e71ad88609fa0a42e13 (127.6.40.66:45523)
I20260806 22:47:07.115366 11325 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.40.66@KRBTEST.COM'} at 127.6.40.66:43131
WARNING: no policy specified for kudu/127.6.40.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.40.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.40.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.67.keytab.
Entry for principal kudu/127.6.40.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.67.keytab.
I20260806 22:47:07.139901 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.67:0
--local_ip_for_outbound_sockets=127.6.40.67
--webserver_interface=127.6.40.67
--webserver_port=0
--tserver_master_addrs=127.6.40.126:37687
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.67.keytab
--principal=kudu/127.6.40.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:35965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:47:07.225776 11821 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:47:07.226046 11821 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:47:07.226094 11821 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:47:07.227389 11821 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:47:07.227464 11821 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:47:07.227510 11821 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.67
I20260806 22:47:07.229199 11821 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:35965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.67.keytab
--principal=kudu/127.6.40.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.40.67
--webserver_port=0
--tserver_master_addrs=127.6.40.126:37687
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11821
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.67
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:47:07.229485 11821 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:47:07.229750 11821 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:47:07.235635 11829 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:07.235680 11827 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:07.235675 11832 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:07.235848 11821 server_base.cc:1034] running on GCE node
I20260806 22:47:07.236123 11821 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:47:07.236390 11821 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:47:07.237561 11821 hybrid_clock.cc:648] HybridClock initialized: now 1786056427237541 us; error 42 us; skew 500 ppm
Aug 06 22:47:07 dist-test-slave-zh2d krb5kdc[11261](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056427, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:47:07.240202 11821 init.cc:377] Logged in from keytab as kudu/127.6.40.67@KRBTEST.COM (short username kudu)
I20260806 22:47:07.240708 11821 webserver.cc:460] Webserver started at http://127.6.40.67:40675/ using document root <none> and password file <none>
I20260806 22:47:07.241024 11821 fs_manager.cc:359] Metadata directory not provided
I20260806 22:47:07.241098 11821 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:47:07.241240 11821 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:47:07.242427 11821 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/minicluster-data/ts-2/data/instance:
uuid: "fa9d4dd259114df6a4c0d0cdc2aeedc4"
format_stamp: "Formatted at 2026-08-06 22:47:07 on dist-test-slave-zh2d"
server_key: "b4a3015a34702212b8b1a32e12f32d3c"
server_key_iv: "c81f56eb4168f484291848e59f9979d0"
server_key_version: "encryptionkey@0"
I20260806 22:47:07.242823 11821 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/minicluster-data/ts-2/wal/instance:
uuid: "fa9d4dd259114df6a4c0d0cdc2aeedc4"
format_stamp: "Formatted at 2026-08-06 22:47:07 on dist-test-slave-zh2d"
server_key: "b4a3015a34702212b8b1a32e12f32d3c"
server_key_iv: "c81f56eb4168f484291848e59f9979d0"
server_key_version: "encryptionkey@0"
I20260806 22:47:07.244240 11821 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:47:07.245178 11845 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:07.245368 11821 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:47:07.245481 11821 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/minicluster-data/ts-2/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/minicluster-data/ts-2/wal
uuid: "fa9d4dd259114df6a4c0d0cdc2aeedc4"
format_stamp: "Formatted at 2026-08-06 22:47:07 on dist-test-slave-zh2d"
server_key: "b4a3015a34702212b8b1a32e12f32d3c"
server_key_iv: "c81f56eb4168f484291848e59f9979d0"
server_key_version: "encryptionkey@0"
I20260806 22:47:07.245559 11821 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:47:07.262706 11821 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:47:07.264601 11821 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:47:07.264778 11821 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:47:07.265067 11821 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:47:07.265413 11821 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:47:07.265470 11821 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:07.265507 11821 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:47:07.265530 11821 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:07.272125 11821 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.67:46103
I20260806 22:47:07.272181 12000 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.67:46103 every 8 connection(s)
I20260806 22:47:07.272495 11821 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
I20260806 22:47:07.275466 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 11821
I20260806 22:47:07.275559 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056405724609-6305-0/minicluster-data/ts-2/wal/instance
I20260806 22:47:07.275760 6305 external_mini_cluster.cc:1428] Setting key 9e892b701e5a0838929b890438d90716
Aug 06 22:47:07 dist-test-slave-zh2d krb5kdc[11261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056427, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.67@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
I20260806 22:47:07.282609 11385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:07.273329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.67:41785 (local address 127.6.40.126:37687)
0806 22:47:07.273411 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:07.273413 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:07.273671 (+ 258us) server_negotiation.cc:407] Connection header received
0806 22:47:07.274619 (+ 948us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:07.274620 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:07.274632 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:07.274692 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:07.275374 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:07.276112 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:07.277351 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:07.277561 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:07.277627 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:07.279681 (+ 2054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:07.279682 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:07.279683 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:07.279690 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:07.281204 (+ 1514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:07.281577 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:07.281578 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:07.281579 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:07.281631 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:07.281951 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:07.281952 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:07.281952 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:07.282106 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:07.282165 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:07.282455 (+ 290us) server_negotiation.cc:299] Negotiation successful
0806 22:47:07.282543 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 22:47:07.282796 12003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:07.273285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:37687 (local address 127.6.40.67:41785)
0806 22:47:07.273526 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:07.273575 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:07.274491 (+ 916us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:07.274802 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:07.274813 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:07.275077 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:07.275268 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:07.275273 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:07.276219 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:07.276221 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:07.277269 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:07.277272 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:07.277647 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:07.277648 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:07.277748 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:07.278202 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:07.278210 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:07.279597 (+ 1387us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:47:07.281275 (+ 1678us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:07.281281 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:07.281282 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:07.281497 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:07.281706 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:07.281707 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:07.281707 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:07.281862 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:07.282177 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:47:07.282183 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:47:07.282318 (+ 135us) client_negotiation.cc:769] Sending connection context
0806 22:47:07.282405 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 22:47:07.282575 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":128,"threads_started":1}
I20260806 22:47:07.283318 12001 heartbeater.cc:347] Connected to a master server at 127.6.40.126:37687
I20260806 22:47:07.283447 12001 heartbeater.cc:464] Registering TS with master...
I20260806 22:47:07.283636 12001 heartbeater.cc:511] Master 127.6.40.126:37687 requested a full tablet report, sending...
I20260806 22:47:07.284061 11325 ts_manager.cc:194] Registered new tserver with Master: fa9d4dd259114df6a4c0d0cdc2aeedc4 (127.6.40.67:46103)
I20260806 22:47:07.284469 11325 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.40.67@KRBTEST.COM'} at 127.6.40.67:41785
I20260806 22:47:07.287622 6305 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 22:47:07 dist-test-slave-zh2d krb5kdc[11261](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056427, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 22:47:07 dist-test-slave-zh2d krb5kdc[11261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056427, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.40.65@KRBTEST.COM
I20260806 22:47:07.311887 12010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:07.304114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41742 (local address 127.6.40.65:33567)
0806 22:47:07.304300 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:07.304304 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:07.304314 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:47:07.304340 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:07.304342 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:07.304387 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:07.304495 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:07.305165 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:07.305926 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:07.306672 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:07.306987 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:07.307043 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:07.309318 (+ 2275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:07.309326 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:07.309326 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:07.309332 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:07.310859 (+ 1527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:07.311174 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:07.311177 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:07.311177 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:07.311248 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:07.311396 (+ 148us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:07.311397 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:07.311397 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:07.311519 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:07.311626 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:07.311684 (+ 58us) server_negotiation.cc:299] Negotiation successful
0806 22:47:07.311747 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":91,"threads_started":1}
Aug 06 22:47:07 dist-test-slave-zh2d krb5kdc[11261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056427, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
I20260806 22:47:07.319993 11385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:07.313362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50456 (local address 127.6.40.126:37687)
0806 22:47:07.313443 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:07.313445 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:07.313455 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:47:07.313567 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:07.313568 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:07.313580 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:07.313648 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:07.314128 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:07.314618 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:07.315289 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:07.315498 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:07.315585 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:07.317862 (+ 2277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:07.317864 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:07.317864 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:07.317871 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:07.319051 (+ 1180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:07.319359 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:07.319360 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:07.319360 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:07.319409 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:07.319564 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:07.319564 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:07.319565 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:07.319696 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:07.319787 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:07.319872 (+ 85us) server_negotiation.cc:299] Negotiation successful
0806 22:47:07.319930 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
W20260806 22:47:07.320266 11325 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50456
I20260806 22:47:07.320637 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 11421
I20260806 22:47:07.341256 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 11610
I20260806 22:47:07.350211 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 11821
I20260806 22:47:07.356671 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 11280
2026-08-06T22:47:07Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2126 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:47:07 dist-test-slave-zh2d krb5kdc[12017](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:47:07 dist-test-slave-zh2d krb5kdc[12017](info): set up 2 sockets
Aug 06 22:47:07 dist-test-slave-zh2d krb5kdc[12017](info): commencing operation
krb5kdc: starting...
W20260806 22:47:08.575958 6305 mini_kdc.cc:121] Time spent starting KDC: real 1.191s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/krb5kdc/test-user.keytab.
Aug 06 22:47:08 dist-test-slave-zh2d krb5kdc[12017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056428, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:47:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:47:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.40.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.40.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.40.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab.
Entry for principal kudu/127.6.40.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab.
I20260806 22:47:08.705078 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.126:37279
--webserver_interface=127.6.40.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab
--principal=kudu/127.6.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:35337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.40.126:37279
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:47:08.789966 12037 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:47:08.790201 12037 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:47:08.790241 12037 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:47:08.791391 12037 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:47:08.791455 12037 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:47:08.791479 12037 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:47:08.791512 12037 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:47:08.791544 12037 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:47:08.793118 12037 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:35337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.40.126:37279
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab
--principal=kudu/127.6.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.126:37279
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12037
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:47:08.793396 12037 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:47:08.793623 12037 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:47:08.798524 12045 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:08.798617 12046 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:08.798652 12037 server_base.cc:1034] running on GCE node
W20260806 22:47:08.798585 12048 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:08.798990 12037 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:47:08.799279 12037 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:47:08.800450 12037 hybrid_clock.cc:648] HybridClock initialized: now 1786056428800429 us; error 40 us; skew 500 ppm
Aug 06 22:47:08 dist-test-slave-zh2d krb5kdc[12017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056428, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:47:08.808799 12037 init.cc:377] Logged in from keytab as kudu/127.6.40.126@KRBTEST.COM (short username kudu)
I20260806 22:47:08.809257 12037 fs_manager.cc:359] Metadata directory not provided
I20260806 22:47:08.809365 12037 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:47:08.809538 12037 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:47:08.810708 12037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/minicluster-data/master-0/data/instance:
uuid: "f6e41a0bf25f4156bb4aec7676cf3ec7"
format_stamp: "Formatted at 2026-08-06 22:47:08 on dist-test-slave-zh2d"
server_key: "fa74b1fd804b86bf8d20808d2db11446"
server_key_iv: "267e45517246c03638bb4197ee5ffd62"
server_key_version: "encryptionkey@0"
I20260806 22:47:08.811092 12037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/minicluster-data/master-0/wal/instance:
uuid: "f6e41a0bf25f4156bb4aec7676cf3ec7"
format_stamp: "Formatted at 2026-08-06 22:47:08 on dist-test-slave-zh2d"
server_key: "fa74b1fd804b86bf8d20808d2db11446"
server_key_iv: "267e45517246c03638bb4197ee5ffd62"
server_key_version: "encryptionkey@0"
I20260806 22:47:08.812534 12037 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:47:08.813547 12056 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:08.813743 12037 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 22:47:08.813845 12037 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/minicluster-data/master-0/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/minicluster-data/master-0/wal
uuid: "f6e41a0bf25f4156bb4aec7676cf3ec7"
format_stamp: "Formatted at 2026-08-06 22:47:08 on dist-test-slave-zh2d"
server_key: "fa74b1fd804b86bf8d20808d2db11446"
server_key_iv: "267e45517246c03638bb4197ee5ffd62"
server_key_version: "encryptionkey@0"
I20260806 22:47:08.813926 12037 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:47:08.821583 12037 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:47:08.823360 12037 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:47:08.823515 12037 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:47:08.827620 12037 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.126:37279
I20260806 22:47:08.827646 12142 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.126:37279 every 8 connection(s)
I20260806 22:47:08.827950 12037 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
I20260806 22:47:08.829005 12143 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:47:08.830577 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 12037
I20260806 22:47:08.830706 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/minicluster-data/master-0/wal/instance
I20260806 22:47:08.830914 6305 external_mini_cluster.cc:1428] Setting key d05e9bd7aa61ac95a70aaaa7079b3e6c
I20260806 22:47:08.831580 12143 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6e41a0bf25f4156bb4aec7676cf3ec7: Bootstrap starting.
I20260806 22:47:08.832321 12143 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f6e41a0bf25f4156bb4aec7676cf3ec7: Neither blocks nor log segments found. Creating new log.
I20260806 22:47:08.832664 12143 log.cc:824] T 00000000000000000000000000000000 P f6e41a0bf25f4156bb4aec7676cf3ec7: Log is configured to *not* fsync() on all Append() calls
I20260806 22:47:08.833431 12143 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6e41a0bf25f4156bb4aec7676cf3ec7: No bootstrap required, opened a new log
I20260806 22:47:08.835686 12143 raft_consensus.cc:348] T 00000000000000000000000000000000 P f6e41a0bf25f4156bb4aec7676cf3ec7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6e41a0bf25f4156bb4aec7676cf3ec7" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 37279 } }
I20260806 22:47:08.835851 12143 raft_consensus.cc:374] T 00000000000000000000000000000000 P f6e41a0bf25f4156bb4aec7676cf3ec7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:47:08.835884 12143 raft_consensus.cc:729] T 00000000000000000000000000000000 P f6e41a0bf25f4156bb4aec7676cf3ec7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e41a0bf25f4156bb4aec7676cf3ec7, State: Initialized, Role: FOLLOWER
I20260806 22:47:08.836032 12143 consensus_queue.cc:260] T 00000000000000000000000000000000 P f6e41a0bf25f4156bb4aec7676cf3ec7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6e41a0bf25f4156bb4aec7676cf3ec7" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 37279 } }
I20260806 22:47:08.836119 12143 raft_consensus.cc:388] T 00000000000000000000000000000000 P f6e41a0bf25f4156bb4aec7676cf3ec7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:47:08.836170 12143 raft_consensus.cc:482] T 00000000000000000000000000000000 P f6e41a0bf25f4156bb4aec7676cf3ec7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:47:08.836222 12143 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f6e41a0bf25f4156bb4aec7676cf3ec7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:47:08.836747 12143 raft_consensus.cc:504] T 00000000000000000000000000000000 P f6e41a0bf25f4156bb4aec7676cf3ec7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6e41a0bf25f4156bb4aec7676cf3ec7" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 37279 } }
I20260806 22:47:08.836889 12143 leader_election.cc:302] T 00000000000000000000000000000000 P f6e41a0bf25f4156bb4aec7676cf3ec7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f6e41a0bf25f4156bb4aec7676cf3ec7; no voters:
I20260806 22:47:08.837062 12143 leader_election.cc:288] T 00000000000000000000000000000000 P f6e41a0bf25f4156bb4aec7676cf3ec7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:47:08.837087 12150 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f6e41a0bf25f4156bb4aec7676cf3ec7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:47:08.837211 12150 raft_consensus.cc:686] T 00000000000000000000000000000000 P f6e41a0bf25f4156bb4aec7676cf3ec7 [term 1 LEADER]: Becoming Leader. State: Replica: f6e41a0bf25f4156bb4aec7676cf3ec7, State: Running, Role: LEADER
I20260806 22:47:08.837293 12150 consensus_queue.cc:237] T 00000000000000000000000000000000 P f6e41a0bf25f4156bb4aec7676cf3ec7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6e41a0bf25f4156bb4aec7676cf3ec7" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 37279 } }
I20260806 22:47:08.837323 12143 sys_catalog.cc:564] T 00000000000000000000000000000000 P f6e41a0bf25f4156bb4aec7676cf3ec7 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:47:08.837615 12150 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6e41a0bf25f4156bb4aec7676cf3ec7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f6e41a0bf25f4156bb4aec7676cf3ec7. Latest consensus state: current_term: 1 leader_uuid: "f6e41a0bf25f4156bb4aec7676cf3ec7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6e41a0bf25f4156bb4aec7676cf3ec7" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 37279 } } }
I20260806 22:47:08.837690 12150 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6e41a0bf25f4156bb4aec7676cf3ec7 [sys.catalog]: This master's current role is: LEADER
I20260806 22:47:08.837631 12154 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6e41a0bf25f4156bb4aec7676cf3ec7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f6e41a0bf25f4156bb4aec7676cf3ec7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6e41a0bf25f4156bb4aec7676cf3ec7" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 37279 } } }
I20260806 22:47:08.837853 12154 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6e41a0bf25f4156bb4aec7676cf3ec7 [sys.catalog]: This master's current role is: LEADER
I20260806 22:47:08.838030 12158 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 06 22:47:08 dist-test-slave-zh2d krb5kdc[12017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056428, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
I20260806 22:47:08.838547 12158 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:47:08.840452 12158 catalog_manager.cc:1347] Generated new cluster ID: 2337e6db221a49beab6f63acf14a56e3
I20260806 22:47:08.840509 12158 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:47:08.842653 12147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:08.831450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51588 (local address 127.6.40.126:37279)
0806 22:47:08.831641 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:08.831650 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:08.831683 (+ 33us) server_negotiation.cc:407] Connection header received
0806 22:47:08.832287 (+ 604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:08.832297 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:08.832360 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:08.832533 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:08.833258 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:08.834601 (+ 1343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:08.835338 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:08.835952 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:08.836063 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:08.838950 (+ 2887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:08.838954 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:08.838954 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:08.838966 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:08.841086 (+ 2120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:08.841430 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:08.841433 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:08.841433 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:08.841534 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:08.841758 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:08.841759 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:08.841759 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:08.842047 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:08.842180 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:08.842327 (+ 147us) server_negotiation.cc:299] Negotiation successful
0806 22:47:08.842456 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":89,"threads_started":1}
I20260806 22:47:08.845671 12158 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:47:08.846414 12158 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:47:08.856711 12158 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f6e41a0bf25f4156bb4aec7676cf3ec7: Generated new TSK 0
I20260806 22:47:08.856923 12158 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.40.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.40.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.40.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.65.keytab.
Entry for principal kudu/127.6.40.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.65.keytab.
I20260806 22:47:08.887801 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.65:0
--local_ip_for_outbound_sockets=127.6.40.65
--webserver_interface=127.6.40.65
--webserver_port=0
--tserver_master_addrs=127.6.40.126:37279
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.65.keytab
--principal=kudu/127.6.40.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:35337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:47:08.976730 12181 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:47:08.977025 12181 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:47:08.977073 12181 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:47:08.978240 12181 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:47:08.978307 12181 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:47:08.978350 12181 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.65
I20260806 22:47:08.979866 12181 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:35337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.65.keytab
--principal=kudu/127.6.40.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.40.65
--webserver_port=0
--tserver_master_addrs=127.6.40.126:37279
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12181
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.65
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:47:08.980132 12181 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:47:08.980370 12181 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:47:08.985615 12190 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:08.985615 12193 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:08.985611 12189 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:08.985855 12181 server_base.cc:1034] running on GCE node
I20260806 22:47:08.986142 12181 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:47:08.986435 12181 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:47:08.987651 12181 hybrid_clock.cc:648] HybridClock initialized: now 1786056428987635 us; error 27 us; skew 500 ppm
Aug 06 22:47:08 dist-test-slave-zh2d krb5kdc[12017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056428, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:47:08.990897 12181 init.cc:377] Logged in from keytab as kudu/127.6.40.65@KRBTEST.COM (short username kudu)
I20260806 22:47:08.991300 12181 fs_manager.cc:359] Metadata directory not provided
I20260806 22:47:08.991374 12181 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:47:08.991515 12181 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:47:08.993069 12181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/minicluster-data/ts-0/data/instance:
uuid: "7f1e18ca997f48d5aaaa76647750d809"
format_stamp: "Formatted at 2026-08-06 22:47:08 on dist-test-slave-zh2d"
server_key: "5209827a66934dc91490f24d97b619db"
server_key_iv: "b9be77d165875a6604ebdb7e83bc7626"
server_key_version: "encryptionkey@0"
I20260806 22:47:08.993521 12181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/minicluster-data/ts-0/wal/instance:
uuid: "7f1e18ca997f48d5aaaa76647750d809"
format_stamp: "Formatted at 2026-08-06 22:47:08 on dist-test-slave-zh2d"
server_key: "5209827a66934dc91490f24d97b619db"
server_key_iv: "b9be77d165875a6604ebdb7e83bc7626"
server_key_version: "encryptionkey@0"
I20260806 22:47:08.995488 12181 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260806 22:47:08.996479 12200 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:08.996702 12181 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:47:08.996805 12181 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/minicluster-data/ts-0/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/minicluster-data/ts-0/wal
uuid: "7f1e18ca997f48d5aaaa76647750d809"
format_stamp: "Formatted at 2026-08-06 22:47:08 on dist-test-slave-zh2d"
server_key: "5209827a66934dc91490f24d97b619db"
server_key_iv: "b9be77d165875a6604ebdb7e83bc7626"
server_key_version: "encryptionkey@0"
I20260806 22:47:08.996910 12181 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:47:09.014638 12181 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:47:09.017593 12181 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:47:09.017824 12181 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:47:09.018153 12181 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:47:09.018672 12181 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:47:09.018744 12181 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:09.018782 12181 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:47:09.018805 12181 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:09.026922 12181 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.65:35805
I20260806 22:47:09.026964 12361 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.65:35805 every 8 connection(s)
I20260806 22:47:09.027290 12181 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
Aug 06 22:47:09 dist-test-slave-zh2d krb5kdc[12017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056428, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.65@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
I20260806 22:47:09.036052 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 12181
I20260806 22:47:09.036182 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/minicluster-data/ts-0/wal/instance
I20260806 22:47:09.036464 6305 external_mini_cluster.cc:1428] Setting key 7823a8504cb967e33ebad867bd9c33f1
I20260806 22:47:09.038784 12147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:09.028093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.65:42257 (local address 127.6.40.126:37279)
0806 22:47:09.028161 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:09.028164 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:09.028392 (+ 228us) server_negotiation.cc:407] Connection header received
0806 22:47:09.028914 (+ 522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:09.028916 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:09.028931 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:09.029019 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:09.029959 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:09.030801 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:09.031849 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:09.032237 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:09.032313 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:09.035093 (+ 2780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:09.035095 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:09.035095 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:09.035102 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:09.036942 (+ 1840us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:09.037577 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:09.037579 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:09.037580 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:09.037643 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:09.037979 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:09.037980 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:09.037981 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:09.038142 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:09.038222 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:09.038583 (+ 361us) server_negotiation.cc:299] Negotiation successful
0806 22:47:09.038689 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 22:47:09.039005 12365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:09.028063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:37279 (local address 127.6.40.65:42257)
0806 22:47:09.028251 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:09.028296 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:09.028788 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:09.029076 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:09.029080 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:09.029349 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:09.029859 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:09.029864 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:09.030907 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:09.030909 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:09.031757 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:09.031759 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:09.032335 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:09.032335 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:09.032438 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:09.033036 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:09.033056 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:09.034914 (+ 1858us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:47:09.037042 (+ 2128us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:09.037051 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:09.037053 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:09.037488 (+ 435us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:09.037711 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:09.037712 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:09.037712 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:09.037855 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:09.038225 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:47:09.038233 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:47:09.038433 (+ 200us) client_negotiation.cc:769] Sending connection context
0806 22:47:09.038559 (+ 126us) client_negotiation.cc:240] Negotiation successful
0806 22:47:09.038727 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":108,"threads_started":1}
I20260806 22:47:09.039496 12363 heartbeater.cc:347] Connected to a master server at 127.6.40.126:37279
I20260806 22:47:09.039597 12363 heartbeater.cc:464] Registering TS with master...
I20260806 22:47:09.039759 12363 heartbeater.cc:511] Master 127.6.40.126:37279 requested a full tablet report, sending...
I20260806 22:47:09.040328 12082 ts_manager.cc:194] Registered new tserver with Master: 7f1e18ca997f48d5aaaa76647750d809 (127.6.40.65:35805)
I20260806 22:47:09.041301 12082 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.40.65@KRBTEST.COM'} at 127.6.40.65:42257
WARNING: no policy specified for kudu/127.6.40.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.40.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.40.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.66.keytab.
Entry for principal kudu/127.6.40.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.66.keytab.
I20260806 22:47:09.072052 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.66:0
--local_ip_for_outbound_sockets=127.6.40.66
--webserver_interface=127.6.40.66
--webserver_port=0
--tserver_master_addrs=127.6.40.126:37279
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.66.keytab
--principal=kudu/127.6.40.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:35337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:47:09.157409 12369 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:47:09.157653 12369 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:47:09.157693 12369 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:47:09.158857 12369 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:47:09.158926 12369 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:47:09.158977 12369 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.66
I20260806 22:47:09.160501 12369 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:35337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.66.keytab
--principal=kudu/127.6.40.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.40.66
--webserver_port=0
--tserver_master_addrs=127.6.40.126:37279
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12369
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.66
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:47:09.160818 12369 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:47:09.161093 12369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:47:09.166296 12378 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:09.166267 12377 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:09.166314 12380 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:09.166347 12369 server_base.cc:1034] running on GCE node
I20260806 22:47:09.166692 12369 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:47:09.166960 12369 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:47:09.168115 12369 hybrid_clock.cc:648] HybridClock initialized: now 1786056429168088 us; error 39 us; skew 500 ppm
Aug 06 22:47:09 dist-test-slave-zh2d krb5kdc[12017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056429, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:47:09.170823 12369 init.cc:377] Logged in from keytab as kudu/127.6.40.66@KRBTEST.COM (short username kudu)
I20260806 22:47:09.171200 12369 fs_manager.cc:359] Metadata directory not provided
I20260806 22:47:09.171277 12369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:47:09.171433 12369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:47:09.172523 12369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/minicluster-data/ts-1/data/instance:
uuid: "dddde889c8ea4ccea123722b2637f25d"
format_stamp: "Formatted at 2026-08-06 22:47:09 on dist-test-slave-zh2d"
server_key: "f50d2e210193a094d5ddfbdbfb67d914"
server_key_iv: "3123246d3a37b8abe4ab433e30956778"
server_key_version: "encryptionkey@0"
I20260806 22:47:09.172955 12369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/minicluster-data/ts-1/wal/instance:
uuid: "dddde889c8ea4ccea123722b2637f25d"
format_stamp: "Formatted at 2026-08-06 22:47:09 on dist-test-slave-zh2d"
server_key: "f50d2e210193a094d5ddfbdbfb67d914"
server_key_iv: "3123246d3a37b8abe4ab433e30956778"
server_key_version: "encryptionkey@0"
I20260806 22:47:09.174297 12369 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:47:09.175036 12388 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:09.175204 12369 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:47:09.175307 12369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/minicluster-data/ts-1/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/minicluster-data/ts-1/wal
uuid: "dddde889c8ea4ccea123722b2637f25d"
format_stamp: "Formatted at 2026-08-06 22:47:09 on dist-test-slave-zh2d"
server_key: "f50d2e210193a094d5ddfbdbfb67d914"
server_key_iv: "3123246d3a37b8abe4ab433e30956778"
server_key_version: "encryptionkey@0"
I20260806 22:47:09.175390 12369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:47:09.194237 12369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:47:09.196789 12369 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:47:09.197016 12369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:47:09.197284 12369 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:47:09.197638 12369 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:47:09.197695 12369 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:09.197736 12369 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:47:09.197781 12369 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:09.204072 12369 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.66:37137
I20260806 22:47:09.204154 12548 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.66:37137 every 8 connection(s)
I20260806 22:47:09.204442 12369 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
I20260806 22:47:09.208132 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 12369
I20260806 22:47:09.208220 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/minicluster-data/ts-1/wal/instance
I20260806 22:47:09.208426 6305 external_mini_cluster.cc:1428] Setting key df27040b2bb98abefff7d1f1d14df33e
Aug 06 22:47:09 dist-test-slave-zh2d krb5kdc[12017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056429, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.66@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
I20260806 22:47:09.214690 12147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:09.205255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.66:56593 (local address 127.6.40.126:37279)
0806 22:47:09.205342 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:09.205345 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:09.205544 (+ 199us) server_negotiation.cc:407] Connection header received
0806 22:47:09.206054 (+ 510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:09.206055 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:09.206066 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:09.206120 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:09.207078 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:09.207587 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:09.208783 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:09.209062 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:09.209148 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:09.211480 (+ 2332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:09.211482 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:09.211482 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:09.211493 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:09.213216 (+ 1723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:09.213661 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:09.213663 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:09.213663 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:09.213730 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:09.213997 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:09.213999 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:09.213999 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:09.214143 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:09.214226 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:09.214497 (+ 271us) server_negotiation.cc:299] Negotiation successful
0806 22:47:09.214589 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 22:47:09.214910 12554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:09.205221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:37279 (local address 127.6.40.66:56593)
0806 22:47:09.205414 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:09.205453 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:09.205966 (+ 513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:09.206176 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:09.206180 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:09.206447 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:09.206994 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:09.206999 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:09.207673 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:09.207673 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:09.208681 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:09.208686 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:09.209187 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:09.209188 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:09.209339 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:09.209928 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:09.209937 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:09.211369 (+ 1432us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:47:09.213310 (+ 1941us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:09.213317 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:09.213319 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:09.213575 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:09.213802 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:09.213803 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:09.213803 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:09.213906 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:09.214216 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:47:09.214225 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:47:09.214360 (+ 135us) client_negotiation.cc:769] Sending connection context
0806 22:47:09.214475 (+ 115us) client_negotiation.cc:240] Negotiation successful
0806 22:47:09.214658 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":104,"threads_started":1}
I20260806 22:47:09.215409 12551 heartbeater.cc:347] Connected to a master server at 127.6.40.126:37279
I20260806 22:47:09.215538 12551 heartbeater.cc:464] Registering TS with master...
I20260806 22:47:09.215731 12551 heartbeater.cc:511] Master 127.6.40.126:37279 requested a full tablet report, sending...
I20260806 22:47:09.216181 12082 ts_manager.cc:194] Registered new tserver with Master: dddde889c8ea4ccea123722b2637f25d (127.6.40.66:37137)
I20260806 22:47:09.216569 12082 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.40.66@KRBTEST.COM'} at 127.6.40.66:56593
WARNING: no policy specified for kudu/127.6.40.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.40.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.40.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.67.keytab.
Entry for principal kudu/127.6.40.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.67.keytab.
I20260806 22:47:09.243592 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.67:0
--local_ip_for_outbound_sockets=127.6.40.67
--webserver_interface=127.6.40.67
--webserver_port=0
--tserver_master_addrs=127.6.40.126:37279
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.67.keytab
--principal=kudu/127.6.40.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:35337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:47:09.327874 12557 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:47:09.328094 12557 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:47:09.328118 12557 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:47:09.329316 12557 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:47:09.329360 12557 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:47:09.329386 12557 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.67
I20260806 22:47:09.330852 12557 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:35337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.67.keytab
--principal=kudu/127.6.40.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.40.67
--webserver_port=0
--tserver_master_addrs=127.6.40.126:37279
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12557
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.67
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:47:09.331081 12557 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:47:09.331278 12557 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:47:09.336426 12564 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:09.336452 12569 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:09.336465 12565 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:09.336620 12557 server_base.cc:1034] running on GCE node
I20260806 22:47:09.336968 12557 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:47:09.337260 12557 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:47:09.338480 12557 hybrid_clock.cc:648] HybridClock initialized: now 1786056429338465 us; error 25 us; skew 500 ppm
Aug 06 22:47:09 dist-test-slave-zh2d krb5kdc[12017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056429, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:47:09.341255 12557 init.cc:377] Logged in from keytab as kudu/127.6.40.67@KRBTEST.COM (short username kudu)
I20260806 22:47:09.341670 12557 fs_manager.cc:359] Metadata directory not provided
I20260806 22:47:09.341742 12557 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:47:09.341882 12557 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:47:09.343123 12557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/minicluster-data/ts-2/data/instance:
uuid: "c4d25f613f9f4ef4b01c59700986301a"
format_stamp: "Formatted at 2026-08-06 22:47:09 on dist-test-slave-zh2d"
server_key: "74953b36c0accca02315a75b09fef8cc"
server_key_iv: "8b3a203bf65be202cfdd9ee428c3563a"
server_key_version: "encryptionkey@0"
I20260806 22:47:09.343499 12557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/minicluster-data/ts-2/wal/instance:
uuid: "c4d25f613f9f4ef4b01c59700986301a"
format_stamp: "Formatted at 2026-08-06 22:47:09 on dist-test-slave-zh2d"
server_key: "74953b36c0accca02315a75b09fef8cc"
server_key_iv: "8b3a203bf65be202cfdd9ee428c3563a"
server_key_version: "encryptionkey@0"
I20260806 22:47:09.344983 12557 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:47:09.345772 12577 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:09.345959 12557 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:47:09.346061 12557 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/minicluster-data/ts-2/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/minicluster-data/ts-2/wal
uuid: "c4d25f613f9f4ef4b01c59700986301a"
format_stamp: "Formatted at 2026-08-06 22:47:09 on dist-test-slave-zh2d"
server_key: "74953b36c0accca02315a75b09fef8cc"
server_key_iv: "8b3a203bf65be202cfdd9ee428c3563a"
server_key_version: "encryptionkey@0"
I20260806 22:47:09.346148 12557 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:47:09.369678 12557 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:47:09.371624 12557 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:47:09.371788 12557 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:47:09.372046 12557 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:47:09.372362 12557 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:47:09.372416 12557 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:09.372469 12557 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:47:09.372501 12557 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:09.379037 12557 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.67:40139
I20260806 22:47:09.379124 12756 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.67:40139 every 8 connection(s)
I20260806 22:47:09.379460 12557 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
Aug 06 22:47:09 dist-test-slave-zh2d krb5kdc[12017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056429, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.67@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
I20260806 22:47:09.388895 12147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:09.380276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.67:52745 (local address 127.6.40.126:37279)
0806 22:47:09.380323 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:09.380325 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:09.380547 (+ 222us) server_negotiation.cc:407] Connection header received
0806 22:47:09.381180 (+ 633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:09.381182 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:09.381197 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:09.381248 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:09.382253 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:09.382764 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:09.383801 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:09.384006 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:09.384092 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:09.386257 (+ 2165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:09.386258 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:09.386259 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:09.386267 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:09.387459 (+ 1192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:09.387867 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:09.387868 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:09.387869 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:09.387933 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:09.388189 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:09.388190 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:09.388191 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:09.388310 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:09.388393 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:09.388681 (+ 288us) server_negotiation.cc:299] Negotiation successful
0806 22:47:09.388765 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 22:47:09.389134 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 12557
I20260806 22:47:09.389256 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/minicluster-data/ts-2/wal/instance
I20260806 22:47:09.389446 6305 external_mini_cluster.cc:1428] Setting key 5ebf111cea86e68a093f8d7123d4d2e6
I20260806 22:47:09.389658 12762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:09.380248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:37279 (local address 127.6.40.67:52745)
0806 22:47:09.380415 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:09.380455 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:09.381061 (+ 606us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:09.381318 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:09.381321 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:09.381616 (+ 295us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:09.382142 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:09.382146 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:09.382866 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:09.382867 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:09.383709 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:09.383711 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:09.384103 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:09.384104 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:09.384233 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:09.384700 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:09.384708 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:09.386153 (+ 1445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:47:09.387551 (+ 1398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:09.387559 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:09.387562 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:09.387787 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:09.388009 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:09.388010 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:09.388010 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:09.388120 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:09.388381 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:47:09.388389 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:47:09.388542 (+ 153us) client_negotiation.cc:769] Sending connection context
0806 22:47:09.388655 (+ 113us) client_negotiation.cc:240] Negotiation successful
0806 22:47:09.388816 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":71,"threads_started":1}
I20260806 22:47:09.390174 12757 heartbeater.cc:347] Connected to a master server at 127.6.40.126:37279
I20260806 22:47:09.390300 12757 heartbeater.cc:464] Registering TS with master...
I20260806 22:47:09.390463 12757 heartbeater.cc:511] Master 127.6.40.126:37279 requested a full tablet report, sending...
I20260806 22:47:09.390873 12082 ts_manager.cc:194] Registered new tserver with Master: c4d25f613f9f4ef4b01c59700986301a (127.6.40.67:40139)
I20260806 22:47:09.391223 12082 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.40.67@KRBTEST.COM'} at 127.6.40.67:52745
I20260806 22:47:09.401275 6305 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:47:09.408504 12147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:09.402584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51600 (local address 127.6.40.126:37279)
0806 22:47:09.402633 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:09.402635 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:09.402741 (+ 106us) server_negotiation.cc:407] Connection header received
0806 22:47:09.402781 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:09.402781 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:09.402790 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:09.402842 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:09.403369 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:09.403847 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:09.404628 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:09.404801 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:09.404902 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:09.406080 (+ 1178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:09.406082 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:09.406082 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:09.406088 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:09.407464 (+ 1376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:09.407793 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:09.407794 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:09.407795 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:09.407845 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:09.408009 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:09.408009 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:09.408010 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:09.408155 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:09.408236 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:09.408380 (+ 144us) server_negotiation.cc:299] Negotiation successful
0806 22:47:09.408441 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 22:47:09.410050 12082 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51600:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:47:09.411636 12082 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:47:09.418617 12775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:09.414051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.65:35805 (local address 127.0.0.1:51950)
0806 22:47:09.414462 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:09.414475 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:09.414554 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:09.414761 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:09.414765 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:09.414767 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:47:09.414840 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:09.414841 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:09.416249 (+ 1408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:09.416250 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:09.417609 (+ 1359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:09.417612 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:09.418307 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:09.418308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:09.418388 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:09.418402 (+ 14us) client_negotiation.cc:769] Sending connection context
0806 22:47:09.418479 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 22:47:09.418540 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"mutex_wait_us":81,"thread_start_us":53,"threads_started":1}
I20260806 22:47:09.418651 12779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:09.414182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51950 (local address 127.6.40.65:35805)
0806 22:47:09.414347 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:09.414352 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:09.414545 (+ 193us) server_negotiation.cc:407] Connection header received
0806 22:47:09.414627 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:09.414629 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:09.414648 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:09.414734 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:47:09.414944 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:09.416124 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:09.417696 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:09.418214 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:09.418305 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:09.418370 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:09.418515 (+ 145us) server_negotiation.cc:299] Negotiation successful
0806 22:47:09.418564 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":91,"threads_started":1}
I20260806 22:47:09.419080 12780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:09.414245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.67:40139 (local address 127.0.0.1:34510)
0806 22:47:09.414645 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:09.414653 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:09.414686 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:09.415158 (+ 472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:09.415159 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:09.415160 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:47:09.415196 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:09.415197 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:09.416075 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:09.416076 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:09.418043 (+ 1967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:09.418046 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:09.418834 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:09.418834 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:09.418895 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:09.418901 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 22:47:09.418966 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 22:47:09.419010 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":384,"mutex_wait_us":52,"thread_start_us":78,"threads_started":1}
I20260806 22:47:09.419099 12782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:09.414477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34510 (local address 127.6.40.67:40139)
0806 22:47:09.414904 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:09.414908 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:09.414918 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:47:09.414968 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:09.414970 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:09.414984 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:09.415067 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:47:09.415279 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:09.415964 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:09.418141 (+ 2177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:09.418750 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:09.418832 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:09.418891 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:09.418978 (+ 87us) server_negotiation.cc:299] Negotiation successful
0806 22:47:09.419018 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":101,"threads_started":1}
I20260806 22:47:09.419564 12777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:09.414051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.66:37137 (local address 127.0.0.1:39946)
0806 22:47:09.414388 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:09.414432 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:09.414522 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:09.414962 (+ 440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:09.414963 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:09.414965 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:47:09.415024 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:09.415026 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:09.416289 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:09.416289 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:09.417597 (+ 1308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:09.417599 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:09.419341 (+ 1742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:09.419342 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:09.419404 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:09.419407 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:47:09.419465 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 22:47:09.419500 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"mutex_wait_us":10,"thread_start_us":76,"threads_started":1}
I20260806 22:47:09.419612 12781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:09.414294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39946 (local address 127.6.40.66:37137)
0806 22:47:09.414463 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:09.414467 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:09.414496 (+ 29us) server_negotiation.cc:407] Connection header received
0806 22:47:09.414643 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:09.414645 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:09.414659 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:09.414860 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:47:09.415091 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:09.416187 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:09.418410 (+ 2223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:09.419181 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:09.419249 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:09.419306 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:09.419485 (+ 179us) server_negotiation.cc:299] Negotiation successful
0806 22:47:09.419529 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":94,"threads_started":1}
I20260806 22:47:09.419719 12266 tablet_service.cc:1467] Processing CreateTablet for tablet bdd951ffbecd4cd3974bd0450e00632d (DEFAULT_TABLE table=test-table [id=19ab5f61671c43afb9f54681f47761da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:47:09.419790 12649 tablet_service.cc:1467] Processing CreateTablet for tablet bdd951ffbecd4cd3974bd0450e00632d (DEFAULT_TABLE table=test-table [id=19ab5f61671c43afb9f54681f47761da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:47:09.420047 12266 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bdd951ffbecd4cd3974bd0450e00632d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:47:09.420058 12649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bdd951ffbecd4cd3974bd0450e00632d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:47:09.421590 12448 tablet_service.cc:1467] Processing CreateTablet for tablet bdd951ffbecd4cd3974bd0450e00632d (DEFAULT_TABLE table=test-table [id=19ab5f61671c43afb9f54681f47761da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:47:09.421904 12448 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bdd951ffbecd4cd3974bd0450e00632d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:47:09.422741 12784 tablet_bootstrap.cc:492] T bdd951ffbecd4cd3974bd0450e00632d P 7f1e18ca997f48d5aaaa76647750d809: Bootstrap starting.
I20260806 22:47:09.423448 12784 tablet_bootstrap.cc:654] T bdd951ffbecd4cd3974bd0450e00632d P 7f1e18ca997f48d5aaaa76647750d809: Neither blocks nor log segments found. Creating new log.
I20260806 22:47:09.424084 12784 log.cc:824] T bdd951ffbecd4cd3974bd0450e00632d P 7f1e18ca997f48d5aaaa76647750d809: Log is configured to *not* fsync() on all Append() calls
I20260806 22:47:09.424794 12784 tablet_bootstrap.cc:492] T bdd951ffbecd4cd3974bd0450e00632d P 7f1e18ca997f48d5aaaa76647750d809: No bootstrap required, opened a new log
I20260806 22:47:09.424896 12784 ts_tablet_manager.cc:1397] T bdd951ffbecd4cd3974bd0450e00632d P 7f1e18ca997f48d5aaaa76647750d809: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:47:09.425158 12787 tablet_bootstrap.cc:492] T bdd951ffbecd4cd3974bd0450e00632d P dddde889c8ea4ccea123722b2637f25d: Bootstrap starting.
I20260806 22:47:09.425359 12785 tablet_bootstrap.cc:492] T bdd951ffbecd4cd3974bd0450e00632d P c4d25f613f9f4ef4b01c59700986301a: Bootstrap starting.
I20260806 22:47:09.425922 12787 tablet_bootstrap.cc:654] T bdd951ffbecd4cd3974bd0450e00632d P dddde889c8ea4ccea123722b2637f25d: Neither blocks nor log segments found. Creating new log.
I20260806 22:47:09.425988 12785 tablet_bootstrap.cc:654] T bdd951ffbecd4cd3974bd0450e00632d P c4d25f613f9f4ef4b01c59700986301a: Neither blocks nor log segments found. Creating new log.
I20260806 22:47:09.426544 12785 log.cc:824] T bdd951ffbecd4cd3974bd0450e00632d P c4d25f613f9f4ef4b01c59700986301a: Log is configured to *not* fsync() on all Append() calls
I20260806 22:47:09.426643 12787 log.cc:824] T bdd951ffbecd4cd3974bd0450e00632d P dddde889c8ea4ccea123722b2637f25d: Log is configured to *not* fsync() on all Append() calls
I20260806 22:47:09.427428 12785 tablet_bootstrap.cc:492] T bdd951ffbecd4cd3974bd0450e00632d P c4d25f613f9f4ef4b01c59700986301a: No bootstrap required, opened a new log
I20260806 22:47:09.427436 12787 tablet_bootstrap.cc:492] T bdd951ffbecd4cd3974bd0450e00632d P dddde889c8ea4ccea123722b2637f25d: No bootstrap required, opened a new log
I20260806 22:47:09.427296 12784 raft_consensus.cc:348] T bdd951ffbecd4cd3974bd0450e00632d P 7f1e18ca997f48d5aaaa76647750d809 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f1e18ca997f48d5aaaa76647750d809" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 35805 } } peers { permanent_uuid: "dddde889c8ea4ccea123722b2637f25d" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 37137 } } peers { permanent_uuid: "c4d25f613f9f4ef4b01c59700986301a" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 40139 } }
I20260806 22:47:09.427498 12784 raft_consensus.cc:374] T bdd951ffbecd4cd3974bd0450e00632d P 7f1e18ca997f48d5aaaa76647750d809 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:47:09.427505 12785 ts_tablet_manager.cc:1397] T bdd951ffbecd4cd3974bd0450e00632d P c4d25f613f9f4ef4b01c59700986301a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:47:09.427523 12787 ts_tablet_manager.cc:1397] T bdd951ffbecd4cd3974bd0450e00632d P dddde889c8ea4ccea123722b2637f25d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:47:09.427547 12784 raft_consensus.cc:729] T bdd951ffbecd4cd3974bd0450e00632d P 7f1e18ca997f48d5aaaa76647750d809 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f1e18ca997f48d5aaaa76647750d809, State: Initialized, Role: FOLLOWER
I20260806 22:47:09.427697 12784 consensus_queue.cc:260] T bdd951ffbecd4cd3974bd0450e00632d P 7f1e18ca997f48d5aaaa76647750d809 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f1e18ca997f48d5aaaa76647750d809" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 35805 } } peers { permanent_uuid: "dddde889c8ea4ccea123722b2637f25d" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 37137 } } peers { permanent_uuid: "c4d25f613f9f4ef4b01c59700986301a" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 40139 } }
I20260806 22:47:09.428002 12784 ts_tablet_manager.cc:1428] T bdd951ffbecd4cd3974bd0450e00632d P 7f1e18ca997f48d5aaaa76647750d809: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:47:09.428154 12363 heartbeater.cc:503] Master 127.6.40.126:37279 was elected leader, sending a full tablet report...
I20260806 22:47:09.429629 12785 raft_consensus.cc:348] T bdd951ffbecd4cd3974bd0450e00632d P c4d25f613f9f4ef4b01c59700986301a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f1e18ca997f48d5aaaa76647750d809" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 35805 } } peers { permanent_uuid: "dddde889c8ea4ccea123722b2637f25d" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 37137 } } peers { permanent_uuid: "c4d25f613f9f4ef4b01c59700986301a" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 40139 } }
I20260806 22:47:09.429756 12785 raft_consensus.cc:374] T bdd951ffbecd4cd3974bd0450e00632d P c4d25f613f9f4ef4b01c59700986301a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:47:09.429777 12785 raft_consensus.cc:729] T bdd951ffbecd4cd3974bd0450e00632d P c4d25f613f9f4ef4b01c59700986301a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4d25f613f9f4ef4b01c59700986301a, State: Initialized, Role: FOLLOWER
I20260806 22:47:09.429899 12785 consensus_queue.cc:260] T bdd951ffbecd4cd3974bd0450e00632d P c4d25f613f9f4ef4b01c59700986301a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f1e18ca997f48d5aaaa76647750d809" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 35805 } } peers { permanent_uuid: "dddde889c8ea4ccea123722b2637f25d" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 37137 } } peers { permanent_uuid: "c4d25f613f9f4ef4b01c59700986301a" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 40139 } }
I20260806 22:47:09.430292 12757 heartbeater.cc:503] Master 127.6.40.126:37279 was elected leader, sending a full tablet report...
I20260806 22:47:09.430202 12787 raft_consensus.cc:348] T bdd951ffbecd4cd3974bd0450e00632d P dddde889c8ea4ccea123722b2637f25d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f1e18ca997f48d5aaaa76647750d809" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 35805 } } peers { permanent_uuid: "dddde889c8ea4ccea123722b2637f25d" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 37137 } } peers { permanent_uuid: "c4d25f613f9f4ef4b01c59700986301a" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 40139 } }
I20260806 22:47:09.430362 12787 raft_consensus.cc:374] T bdd951ffbecd4cd3974bd0450e00632d P dddde889c8ea4ccea123722b2637f25d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:47:09.430397 12787 raft_consensus.cc:729] T bdd951ffbecd4cd3974bd0450e00632d P dddde889c8ea4ccea123722b2637f25d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dddde889c8ea4ccea123722b2637f25d, State: Initialized, Role: FOLLOWER
I20260806 22:47:09.430239 12785 ts_tablet_manager.cc:1428] T bdd951ffbecd4cd3974bd0450e00632d P c4d25f613f9f4ef4b01c59700986301a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:47:09.430529 12787 consensus_queue.cc:260] T bdd951ffbecd4cd3974bd0450e00632d P dddde889c8ea4ccea123722b2637f25d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f1e18ca997f48d5aaaa76647750d809" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 35805 } } peers { permanent_uuid: "dddde889c8ea4ccea123722b2637f25d" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 37137 } } peers { permanent_uuid: "c4d25f613f9f4ef4b01c59700986301a" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 40139 } }
I20260806 22:47:09.430863 12787 ts_tablet_manager.cc:1428] T bdd951ffbecd4cd3974bd0450e00632d P dddde889c8ea4ccea123722b2637f25d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:47:09.430892 12551 heartbeater.cc:503] Master 127.6.40.126:37279 was elected leader, sending a full tablet report...
W20260806 22:47:09.455000 12552 tablet.cc:2369] T bdd951ffbecd4cd3974bd0450e00632d P dddde889c8ea4ccea123722b2637f25d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:47:09.528160 12364 tablet.cc:2369] T bdd951ffbecd4cd3974bd0450e00632d P 7f1e18ca997f48d5aaaa76647750d809: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:47:09.630121 12761 tablet.cc:2369] T bdd951ffbecd4cd3974bd0450e00632d P c4d25f613f9f4ef4b01c59700986301a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:47:09.768684 12796 raft_consensus.cc:482] T bdd951ffbecd4cd3974bd0450e00632d P dddde889c8ea4ccea123722b2637f25d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:47:09.768826 12796 raft_consensus.cc:504] T bdd951ffbecd4cd3974bd0450e00632d P dddde889c8ea4ccea123722b2637f25d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f1e18ca997f48d5aaaa76647750d809" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 35805 } } peers { permanent_uuid: "dddde889c8ea4ccea123722b2637f25d" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 37137 } } peers { permanent_uuid: "c4d25f613f9f4ef4b01c59700986301a" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 40139 } }
I20260806 22:47:09.769320 12796 leader_election.cc:288] T bdd951ffbecd4cd3974bd0450e00632d P dddde889c8ea4ccea123722b2637f25d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7f1e18ca997f48d5aaaa76647750d809 (127.6.40.65:35805), c4d25f613f9f4ef4b01c59700986301a (127.6.40.67:40139)
I20260806 22:47:09.772909 12779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:09.769619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.66:34737 (local address 127.6.40.65:35805)
0806 22:47:09.769675 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:09.769678 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:09.769752 (+ 74us) server_negotiation.cc:407] Connection header received
0806 22:47:09.769825 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:09.769826 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:09.769839 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:09.769918 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:47:09.770092 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:09.770835 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:09.772141 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:09.772506 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:09.772561 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:09.772592 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:09.772760 (+ 168us) server_negotiation.cc:299] Negotiation successful
0806 22:47:09.772811 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 22:47:09.772929 12797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:09.769522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.65:35805 (local address 127.6.40.66:34737)
0806 22:47:09.769673 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:09.769684 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:09.769768 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:09.769922 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:09.769923 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:09.769925 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:47:09.770002 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:09.770004 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:09.770937 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:09.770939 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:09.771986 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:09.771989 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:09.772596 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:09.772597 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:09.772673 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:09.772677 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:47:09.772728 (+ 51us) client_negotiation.cc:240] Negotiation successful
0806 22:47:09.772761 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"mutex_wait_us":19,"thread_start_us":88,"threads_started":1}
I20260806 22:47:09.772974 12782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:09.769615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.66:41797 (local address 127.6.40.67:40139)
0806 22:47:09.769674 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:09.769677 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:09.769753 (+ 76us) server_negotiation.cc:407] Connection header received
0806 22:47:09.769817 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:09.769818 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:09.769830 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:09.769905 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:47:09.770083 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:09.770870 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:09.772087 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:09.772506 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:09.772609 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:09.772620 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:09.772811 (+ 191us) server_negotiation.cc:299] Negotiation successful
0806 22:47:09.772889 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 22:47:09.772938 12800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:09.769515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.67:40139 (local address 127.6.40.66:41797)
0806 22:47:09.769669 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:09.769682 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:09.769755 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:09.769909 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:09.769910 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:09.769912 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:47:09.770009 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:09.770010 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:09.770995 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:09.770996 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:09.771987 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:09.771990 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:09.772627 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:09.772628 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:09.772707 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:09.772730 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 22:47:09.772791 (+ 61us) client_negotiation.cc:240] Negotiation successful
0806 22:47:09.772826 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":74,"threads_started":1}
I20260806 22:47:09.773514 12295 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bdd951ffbecd4cd3974bd0450e00632d" candidate_uuid: "dddde889c8ea4ccea123722b2637f25d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f1e18ca997f48d5aaaa76647750d809" is_pre_election: true
I20260806 22:47:09.773571 12678 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bdd951ffbecd4cd3974bd0450e00632d" candidate_uuid: "dddde889c8ea4ccea123722b2637f25d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4d25f613f9f4ef4b01c59700986301a" is_pre_election: true
I20260806 22:47:09.773684 12295 raft_consensus.cc:2454] T bdd951ffbecd4cd3974bd0450e00632d P 7f1e18ca997f48d5aaaa76647750d809 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dddde889c8ea4ccea123722b2637f25d in term 0.
I20260806 22:47:09.773757 12678 raft_consensus.cc:2454] T bdd951ffbecd4cd3974bd0450e00632d P c4d25f613f9f4ef4b01c59700986301a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dddde889c8ea4ccea123722b2637f25d in term 0.
I20260806 22:47:09.773919 12393 leader_election.cc:302] T bdd951ffbecd4cd3974bd0450e00632d P dddde889c8ea4ccea123722b2637f25d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f1e18ca997f48d5aaaa76647750d809, dddde889c8ea4ccea123722b2637f25d; no voters:
I20260806 22:47:09.774091 12796 raft_consensus.cc:2781] T bdd951ffbecd4cd3974bd0450e00632d P dddde889c8ea4ccea123722b2637f25d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:47:09.774155 12796 raft_consensus.cc:482] T bdd951ffbecd4cd3974bd0450e00632d P dddde889c8ea4ccea123722b2637f25d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:47:09.774173 12796 raft_consensus.cc:3037] T bdd951ffbecd4cd3974bd0450e00632d P dddde889c8ea4ccea123722b2637f25d [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:47:09.774853 12796 raft_consensus.cc:504] T bdd951ffbecd4cd3974bd0450e00632d P dddde889c8ea4ccea123722b2637f25d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f1e18ca997f48d5aaaa76647750d809" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 35805 } } peers { permanent_uuid: "dddde889c8ea4ccea123722b2637f25d" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 37137 } } peers { permanent_uuid: "c4d25f613f9f4ef4b01c59700986301a" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 40139 } }
I20260806 22:47:09.775009 12796 leader_election.cc:288] T bdd951ffbecd4cd3974bd0450e00632d P dddde889c8ea4ccea123722b2637f25d [CANDIDATE]: Term 1 election: Requested vote from peers 7f1e18ca997f48d5aaaa76647750d809 (127.6.40.65:35805), c4d25f613f9f4ef4b01c59700986301a (127.6.40.67:40139)
I20260806 22:47:09.775166 12295 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bdd951ffbecd4cd3974bd0450e00632d" candidate_uuid: "dddde889c8ea4ccea123722b2637f25d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f1e18ca997f48d5aaaa76647750d809"
I20260806 22:47:09.775184 12678 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bdd951ffbecd4cd3974bd0450e00632d" candidate_uuid: "dddde889c8ea4ccea123722b2637f25d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4d25f613f9f4ef4b01c59700986301a"
I20260806 22:47:09.775254 12295 raft_consensus.cc:3037] T bdd951ffbecd4cd3974bd0450e00632d P 7f1e18ca997f48d5aaaa76647750d809 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:47:09.775256 12678 raft_consensus.cc:3037] T bdd951ffbecd4cd3974bd0450e00632d P c4d25f613f9f4ef4b01c59700986301a [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:47:09.776233 12678 raft_consensus.cc:2454] T bdd951ffbecd4cd3974bd0450e00632d P c4d25f613f9f4ef4b01c59700986301a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dddde889c8ea4ccea123722b2637f25d in term 1.
I20260806 22:47:09.776233 12295 raft_consensus.cc:2454] T bdd951ffbecd4cd3974bd0450e00632d P 7f1e18ca997f48d5aaaa76647750d809 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dddde889c8ea4ccea123722b2637f25d in term 1.
I20260806 22:47:09.776424 12392 leader_election.cc:302] T bdd951ffbecd4cd3974bd0450e00632d P dddde889c8ea4ccea123722b2637f25d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4d25f613f9f4ef4b01c59700986301a, dddde889c8ea4ccea123722b2637f25d; no voters:
I20260806 22:47:09.776553 12796 raft_consensus.cc:2781] T bdd951ffbecd4cd3974bd0450e00632d P dddde889c8ea4ccea123722b2637f25d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:47:09.776698 12796 raft_consensus.cc:686] T bdd951ffbecd4cd3974bd0450e00632d P dddde889c8ea4ccea123722b2637f25d [term 1 LEADER]: Becoming Leader. State: Replica: dddde889c8ea4ccea123722b2637f25d, State: Running, Role: LEADER
I20260806 22:47:09.776808 12796 consensus_queue.cc:237] T bdd951ffbecd4cd3974bd0450e00632d P dddde889c8ea4ccea123722b2637f25d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f1e18ca997f48d5aaaa76647750d809" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 35805 } } peers { permanent_uuid: "dddde889c8ea4ccea123722b2637f25d" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 37137 } } peers { permanent_uuid: "c4d25f613f9f4ef4b01c59700986301a" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 40139 } }
I20260806 22:47:09.777698 12082 catalog_manager.cc:5495] T bdd951ffbecd4cd3974bd0450e00632d P dddde889c8ea4ccea123722b2637f25d reported cstate change: term changed from 0 to 1, leader changed from <none> to dddde889c8ea4ccea123722b2637f25d (127.6.40.66). New cstate: current_term: 1 leader_uuid: "dddde889c8ea4ccea123722b2637f25d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f1e18ca997f48d5aaaa76647750d809" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 35805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dddde889c8ea4ccea123722b2637f25d" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 37137 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c4d25f613f9f4ef4b01c59700986301a" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 40139 } health_report { overall_health: UNKNOWN } } }
I20260806 22:47:09.863116 12781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:09.860594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39960 (local address 127.6.40.66:37137)
0806 22:47:09.860669 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:09.860671 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:09.860693 (+ 22us) server_negotiation.cc:407] Connection header received
0806 22:47:09.860744 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:09.860745 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:09.860766 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:09.860843 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:47:09.861018 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:09.861507 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:09.862287 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:09.862501 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:09.862584 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:09.862762 (+ 178us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:47:09.862871 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:47:09.862978 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:09.862997 (+ 19us) server_negotiation.cc:299] Negotiation successful
0806 22:47:09.863017 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 22:47:09.865092 12295 raft_consensus.cc:1261] T bdd951ffbecd4cd3974bd0450e00632d P 7f1e18ca997f48d5aaaa76647750d809 [term 1 FOLLOWER]: Refusing update from remote peer dddde889c8ea4ccea123722b2637f25d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:47:09.865119 12678 raft_consensus.cc:1261] T bdd951ffbecd4cd3974bd0450e00632d P c4d25f613f9f4ef4b01c59700986301a [term 1 FOLLOWER]: Refusing update from remote peer dddde889c8ea4ccea123722b2637f25d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:47:09.865474 12796 consensus_queue.cc:1035] T bdd951ffbecd4cd3974bd0450e00632d P dddde889c8ea4ccea123722b2637f25d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4d25f613f9f4ef4b01c59700986301a" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 40139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:47:09.865653 12801 consensus_queue.cc:1035] T bdd951ffbecd4cd3974bd0450e00632d P dddde889c8ea4ccea123722b2637f25d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f1e18ca997f48d5aaaa76647750d809" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 35805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:47:09.868220 12805 mvcc.cc:205] Tried to move back new op lower bound from 7315687136724283392 to 7315687136366579712. Current Snapshot: MvccSnapshot[applied={T|T < 7315687136724283392}]
I20260806 22:47:09.868947 12809 mvcc.cc:205] Tried to move back new op lower bound from 7315687136724283392 to 7315687136366579712. Current Snapshot: MvccSnapshot[applied={T|T < 7315687136724283392}]
I20260806 22:47:09.869279 12806 mvcc.cc:205] Tried to move back new op lower bound from 7315687136724283392 to 7315687136366579712. Current Snapshot: MvccSnapshot[applied={T|T < 7315687136724283392}]
I20260806 22:47:09.870498 12082 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51600:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:47:09.870612 12082 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51600:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:47:09.871438 12082 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f6e41a0bf25f4156bb4aec7676cf3ec7: Sending DeleteTablet for 3 replicas of tablet bdd951ffbecd4cd3974bd0450e00632d
I20260806 22:47:09.871806 12448 tablet_service.cc:1514] Processing DeleteTablet for tablet bdd951ffbecd4cd3974bd0450e00632d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:47:09 UTC) from {username='kudu', principal='kudu/127.6.40.126@KRBTEST.COM'} at 127.0.0.1:39946
I20260806 22:47:09.871817 12266 tablet_service.cc:1514] Processing DeleteTablet for tablet bdd951ffbecd4cd3974bd0450e00632d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:47:09 UTC) from {username='kudu', principal='kudu/127.6.40.126@KRBTEST.COM'} at 127.0.0.1:51950
I20260806 22:47:09.871881 12649 tablet_service.cc:1514] Processing DeleteTablet for tablet bdd951ffbecd4cd3974bd0450e00632d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:47:09 UTC) from {username='kudu', principal='kudu/127.6.40.126@KRBTEST.COM'} at 127.0.0.1:34510
I20260806 22:47:09.872110 12822 tablet_replica.cc:330] T bdd951ffbecd4cd3974bd0450e00632d P 7f1e18ca997f48d5aaaa76647750d809: stopping tablet replica
I20260806 22:47:09.872216 12822 raft_consensus.cc:2229] T bdd951ffbecd4cd3974bd0450e00632d P 7f1e18ca997f48d5aaaa76647750d809 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:47:09.872326 12822 raft_consensus.cc:2258] T bdd951ffbecd4cd3974bd0450e00632d P 7f1e18ca997f48d5aaaa76647750d809 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:47:09.872372 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 12181
I20260806 22:47:09.872488 12823 tablet_replica.cc:330] T bdd951ffbecd4cd3974bd0450e00632d P c4d25f613f9f4ef4b01c59700986301a: stopping tablet replica
I20260806 22:47:09.872577 12823 raft_consensus.cc:2229] T bdd951ffbecd4cd3974bd0450e00632d P c4d25f613f9f4ef4b01c59700986301a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:47:09.872668 12823 raft_consensus.cc:2258] T bdd951ffbecd4cd3974bd0450e00632d P c4d25f613f9f4ef4b01c59700986301a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:47:09.879859 12823 ts_tablet_manager.cc:1905] T bdd951ffbecd4cd3974bd0450e00632d P c4d25f613f9f4ef4b01c59700986301a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:47:09.880967 12821 tablet_replica.cc:330] T bdd951ffbecd4cd3974bd0450e00632d P dddde889c8ea4ccea123722b2637f25d: stopping tablet replica
I20260806 22:47:09.881140 12821 raft_consensus.cc:2229] T bdd951ffbecd4cd3974bd0450e00632d P dddde889c8ea4ccea123722b2637f25d [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:47:09.881314 12821 raft_consensus.cc:2258] T bdd951ffbecd4cd3974bd0450e00632d P dddde889c8ea4ccea123722b2637f25d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:47:09.881834 12821 ts_tablet_manager.cc:1905] T bdd951ffbecd4cd3974bd0450e00632d P dddde889c8ea4ccea123722b2637f25d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:47:09.882161 12823 ts_tablet_manager.cc:1918] T bdd951ffbecd4cd3974bd0450e00632d P c4d25f613f9f4ef4b01c59700986301a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:47:09.882251 12823 log.cc:1196] T bdd951ffbecd4cd3974bd0450e00632d P c4d25f613f9f4ef4b01c59700986301a: Deleting WAL directory at /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/minicluster-data/ts-2/wal/wals/bdd951ffbecd4cd3974bd0450e00632d
I20260806 22:47:09.882517 12823 ts_tablet_manager.cc:1939] T bdd951ffbecd4cd3974bd0450e00632d P c4d25f613f9f4ef4b01c59700986301a: Deleting consensus metadata
I20260806 22:47:09.882972 12060 catalog_manager.cc:4841] TS c4d25f613f9f4ef4b01c59700986301a (127.6.40.67:40139): tablet bdd951ffbecd4cd3974bd0450e00632d (table test-table [id=19ab5f61671c43afb9f54681f47761da]) successfully deleted
I20260806 22:47:09.883834 12821 ts_tablet_manager.cc:1918] T bdd951ffbecd4cd3974bd0450e00632d P dddde889c8ea4ccea123722b2637f25d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:47:09.883912 12821 log.cc:1196] T bdd951ffbecd4cd3974bd0450e00632d P dddde889c8ea4ccea123722b2637f25d: Deleting WAL directory at /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056405724609-6305-0/minicluster-data/ts-1/wal/wals/bdd951ffbecd4cd3974bd0450e00632d
I20260806 22:47:09.884169 12821 ts_tablet_manager.cc:1939] T bdd951ffbecd4cd3974bd0450e00632d P dddde889c8ea4ccea123722b2637f25d: Deleting consensus metadata
I20260806 22:47:09.885916 12060 catalog_manager.cc:4841] TS dddde889c8ea4ccea123722b2637f25d (127.6.40.66:37137): tablet bdd951ffbecd4cd3974bd0450e00632d (table test-table [id=19ab5f61671c43afb9f54681f47761da]) successfully deleted
W20260806 22:47:09.886209 12061 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.40.65:35805 (error 108)
I20260806 22:47:09.886891 12775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:09.886739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.65:35805 (local address 127.0.0.1:51956)
0806 22:47:09.886804 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:09.886849 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.65:35805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260806 22:47:09.886924 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 12369
W20260806 22:47:09.887076 12061 catalog_manager.cc:4584] TS 7f1e18ca997f48d5aaaa76647750d809 (127.6.40.65:35805): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bdd951ffbecd4cd3974bd0450e00632d: Network error: Client connection negotiation failed: client connection to 127.6.40.65:35805: connect: Connection refused (error 111)
I20260806 22:47:09.895325 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 12557
I20260806 22:47:09.905227 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 12037
2026-08-06T22:47:09Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2547 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-06T22:47:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:47:09Z Disabled control of system clock
I20260806 22:47:09.940687 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056405724609-6305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056405724609-6305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056405724609-6305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.126:35555
--webserver_interface=127.6.40.126
--webserver_port=0
--builtin_ntp_servers=127.6.40.84:45121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.40.126:35555
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 22:47:10.023979 12832 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:47:10.024187 12832 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:47:10.024209 12832 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:47:10.025405 12832 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:47:10.025449 12832 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:47:10.025460 12832 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:47:10.025473 12832 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:47:10.025485 12832 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:47:10.026767 12832 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:45121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056405724609-6305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056405724609-6305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.40.126:35555
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.126:35555
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12832
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056405724609-6305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:47:10.026959 12832 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:47:10.027197 12832 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:47:10.032090 12845 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:10.032122 12840 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:10.032112 12842 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:10.032344 12832 server_base.cc:1034] running on GCE node
I20260806 22:47:10.032662 12832 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:47:10.033008 12832 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:47:10.034205 12832 hybrid_clock.cc:648] HybridClock initialized: now 1786056430034194 us; error 28 us; skew 500 ppm
I20260806 22:47:10.035643 12832 webserver.cc:460] Webserver started at http://127.6.40.126:44287/ using document root <none> and password file <none>
I20260806 22:47:10.035913 12832 fs_manager.cc:359] Metadata directory not provided
I20260806 22:47:10.035964 12832 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:47:10.036126 12832 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:47:10.037305 12832 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056405724609-6305-0/minicluster-data/master-0/data/instance:
uuid: "073e1a46d5b9473380522d27c66148df"
format_stamp: "Formatted at 2026-08-06 22:47:10 on dist-test-slave-zh2d"
server_key: "81f12a0cfbc7a40925a4bf380ab838b7"
server_key_iv: "e16b236a88a00a05a35c8d16f3acc260"
server_key_version: "encryptionkey@0"
I20260806 22:47:10.037779 12832 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056405724609-6305-0/minicluster-data/master-0/wal/instance:
uuid: "073e1a46d5b9473380522d27c66148df"
format_stamp: "Formatted at 2026-08-06 22:47:10 on dist-test-slave-zh2d"
server_key: "81f12a0cfbc7a40925a4bf380ab838b7"
server_key_iv: "e16b236a88a00a05a35c8d16f3acc260"
server_key_version: "encryptionkey@0"
I20260806 22:47:10.039214 12832 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:47:10.039955 12855 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:10.040172 12832 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:47:10.040271 12832 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056405724609-6305-0/minicluster-data/master-0/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056405724609-6305-0/minicluster-data/master-0/wal
uuid: "073e1a46d5b9473380522d27c66148df"
format_stamp: "Formatted at 2026-08-06 22:47:10 on dist-test-slave-zh2d"
server_key: "81f12a0cfbc7a40925a4bf380ab838b7"
server_key_iv: "e16b236a88a00a05a35c8d16f3acc260"
server_key_version: "encryptionkey@0"
I20260806 22:47:10.040370 12832 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056405724609-6305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056405724609-6305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056405724609-6305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:47:10.043876 12832 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:47:10.044145 12832 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:47:10.044286 12832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:47:10.048569 12832 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.126:35555
I20260806 22:47:10.048602 12931 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.126:35555 every 8 connection(s)
I20260806 22:47:10.048988 12832 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
I20260806 22:47:10.049642 12932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:47:10.052716 12932 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 073e1a46d5b9473380522d27c66148df: Bootstrap starting.
I20260806 22:47:10.053406 12932 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 073e1a46d5b9473380522d27c66148df: Neither blocks nor log segments found. Creating new log.
I20260806 22:47:10.053746 12932 log.cc:824] T 00000000000000000000000000000000 P 073e1a46d5b9473380522d27c66148df: Log is configured to *not* fsync() on all Append() calls
I20260806 22:47:10.054492 12932 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 073e1a46d5b9473380522d27c66148df: No bootstrap required, opened a new log
I20260806 22:47:10.055168 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 12832
I20260806 22:47:10.055255 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056405724609-6305-0/minicluster-data/master-0/wal/instance
I20260806 22:47:10.055445 6305 external_mini_cluster.cc:1428] Setting key abdb0026d1ed8e230f8e95122092129d
I20260806 22:47:10.056769 12932 raft_consensus.cc:348] T 00000000000000000000000000000000 P 073e1a46d5b9473380522d27c66148df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "073e1a46d5b9473380522d27c66148df" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 35555 } }
I20260806 22:47:10.056984 12932 raft_consensus.cc:374] T 00000000000000000000000000000000 P 073e1a46d5b9473380522d27c66148df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:47:10.057029 12932 raft_consensus.cc:729] T 00000000000000000000000000000000 P 073e1a46d5b9473380522d27c66148df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 073e1a46d5b9473380522d27c66148df, State: Initialized, Role: FOLLOWER
I20260806 22:47:10.057192 12932 consensus_queue.cc:260] T 00000000000000000000000000000000 P 073e1a46d5b9473380522d27c66148df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "073e1a46d5b9473380522d27c66148df" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 35555 } }
I20260806 22:47:10.057302 12932 raft_consensus.cc:388] T 00000000000000000000000000000000 P 073e1a46d5b9473380522d27c66148df [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:47:10.057350 12932 raft_consensus.cc:482] T 00000000000000000000000000000000 P 073e1a46d5b9473380522d27c66148df [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:47:10.057416 12932 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 073e1a46d5b9473380522d27c66148df [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:47:10.058152 12937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:10.056034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50534 (local address 127.6.40.126:35555)
0806 22:47:10.056272 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:10.056281 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:10.056317 (+ 36us) server_negotiation.cc:407] Connection header received
0806 22:47:10.056725 (+ 408us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:10.056734 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:10.056785 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:10.056969 (+ 184us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:10.057679 (+ 710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:10.057683 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:10.057683 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:47:10.057688 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:10.057736 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:47:10.057752 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:10.057829 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:10.057880 (+ 51us) server_negotiation.cc:299] Negotiation successful
0806 22:47:10.057941 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":87,"threads_started":1}
I20260806 22:47:10.058224 12932 raft_consensus.cc:504] T 00000000000000000000000000000000 P 073e1a46d5b9473380522d27c66148df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "073e1a46d5b9473380522d27c66148df" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 35555 } }
I20260806 22:47:10.058323 12932 leader_election.cc:302] T 00000000000000000000000000000000 P 073e1a46d5b9473380522d27c66148df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 073e1a46d5b9473380522d27c66148df; no voters:
I20260806 22:47:10.058495 12932 leader_election.cc:288] T 00000000000000000000000000000000 P 073e1a46d5b9473380522d27c66148df [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:47:10.058749 12932 sys_catalog.cc:564] T 00000000000000000000000000000000 P 073e1a46d5b9473380522d27c66148df [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:47:10.058842 12938 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 073e1a46d5b9473380522d27c66148df [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:47:10.059005 12938 raft_consensus.cc:686] T 00000000000000000000000000000000 P 073e1a46d5b9473380522d27c66148df [term 1 LEADER]: Becoming Leader. State: Replica: 073e1a46d5b9473380522d27c66148df, State: Running, Role: LEADER
I20260806 22:47:10.059099 12938 consensus_queue.cc:237] T 00000000000000000000000000000000 P 073e1a46d5b9473380522d27c66148df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "073e1a46d5b9473380522d27c66148df" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 35555 } }
I20260806 22:47:10.059545 12939 sys_catalog.cc:455] T 00000000000000000000000000000000 P 073e1a46d5b9473380522d27c66148df [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "073e1a46d5b9473380522d27c66148df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "073e1a46d5b9473380522d27c66148df" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 35555 } } }
I20260806 22:47:10.059562 12940 sys_catalog.cc:455] T 00000000000000000000000000000000 P 073e1a46d5b9473380522d27c66148df [sys.catalog]: SysCatalogTable state changed. Reason: New leader 073e1a46d5b9473380522d27c66148df. Latest consensus state: current_term: 1 leader_uuid: "073e1a46d5b9473380522d27c66148df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "073e1a46d5b9473380522d27c66148df" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 35555 } } }
I20260806 22:47:10.059640 12939 sys_catalog.cc:458] T 00000000000000000000000000000000 P 073e1a46d5b9473380522d27c66148df [sys.catalog]: This master's current role is: LEADER
I20260806 22:47:10.059657 12940 sys_catalog.cc:458] T 00000000000000000000000000000000 P 073e1a46d5b9473380522d27c66148df [sys.catalog]: This master's current role is: LEADER
I20260806 22:47:10.059834 12949 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:47:10.060338 12949 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:47:10.061965 12949 catalog_manager.cc:1347] Generated new cluster ID: e62606e9a6ed419c96ed5800630f4a91
I20260806 22:47:10.062019 12949 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:47:10.073710 12949 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:47:10.073881 12949 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:47:10.077498 12949 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 073e1a46d5b9473380522d27c66148df: Generated new TSK 0
I20260806 22:47:10.077685 12949 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:47:10.079149 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056405724609-6305-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056405724609-6305-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056405724609-6305-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.65:0
--local_ip_for_outbound_sockets=127.6.40.65
--webserver_interface=127.6.40.65
--webserver_port=0
--tserver_master_addrs=127.6.40.126:35555
--builtin_ntp_servers=127.6.40.84:45121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 22:47:10.162623 12966 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:47:10.162827 12966 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:47:10.162849 12966 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:47:10.164104 12966 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:47:10.164144 12966 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:47:10.164171 12966 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.65
I20260806 22:47:10.165601 12966 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:45121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056405724609-6305-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056405724609-6305-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.40.65
--webserver_port=0
--tserver_master_addrs=127.6.40.126:35555
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12966
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.65
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056405724609-6305-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:47:10.165831 12966 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:47:10.166157 12966 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:47:10.172034 12975 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:10.172103 12966 server_base.cc:1034] running on GCE node
W20260806 22:47:10.172047 12976 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:10.172307 12978 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:10.172544 12966 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:47:10.172816 12966 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:47:10.174016 12966 hybrid_clock.cc:648] HybridClock initialized: now 1786056430173988 us; error 43 us; skew 500 ppm
I20260806 22:47:10.175493 12966 webserver.cc:460] Webserver started at http://127.6.40.65:34847/ using document root <none> and password file <none>
I20260806 22:47:10.175750 12966 fs_manager.cc:359] Metadata directory not provided
I20260806 22:47:10.175806 12966 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:47:10.175937 12966 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:47:10.177373 12966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056405724609-6305-0/minicluster-data/ts-0/data/instance:
uuid: "443abc52a3ab4fbe9b9512fdf6281945"
format_stamp: "Formatted at 2026-08-06 22:47:10 on dist-test-slave-zh2d"
server_key: "ce3659d76b69640794b98d7dacaaf9bd"
server_key_iv: "1f40f6df7b6aa55216029d36c29fa423"
server_key_version: "encryptionkey@0"
I20260806 22:47:10.177878 12966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056405724609-6305-0/minicluster-data/ts-0/wal/instance:
uuid: "443abc52a3ab4fbe9b9512fdf6281945"
format_stamp: "Formatted at 2026-08-06 22:47:10 on dist-test-slave-zh2d"
server_key: "ce3659d76b69640794b98d7dacaaf9bd"
server_key_iv: "1f40f6df7b6aa55216029d36c29fa423"
server_key_version: "encryptionkey@0"
I20260806 22:47:10.179774 12966 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260806 22:47:10.180797 12987 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:10.181030 12966 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:47:10.181128 12966 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056405724609-6305-0/minicluster-data/ts-0/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056405724609-6305-0/minicluster-data/ts-0/wal
uuid: "443abc52a3ab4fbe9b9512fdf6281945"
format_stamp: "Formatted at 2026-08-06 22:47:10 on dist-test-slave-zh2d"
server_key: "ce3659d76b69640794b98d7dacaaf9bd"
server_key_iv: "1f40f6df7b6aa55216029d36c29fa423"
server_key_version: "encryptionkey@0"
I20260806 22:47:10.181221 12966 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056405724609-6305-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056405724609-6305-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056405724609-6305-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:47:10.186154 12966 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:47:10.186512 12966 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:47:10.186651 12966 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:47:10.186862 12966 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:47:10.187199 12966 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:47:10.187238 12966 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:10.187268 12966 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:47:10.187284 12966 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:10.194746 12966 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.65:41591
I20260806 22:47:10.194815 13144 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.65:41591 every 8 connection(s)
I20260806 22:47:10.195194 12966 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
I20260806 22:47:10.198379 12937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:10.195990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.65:54329 (local address 127.6.40.126:35555)
0806 22:47:10.196036 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:10.196038 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:10.196325 (+ 287us) server_negotiation.cc:407] Connection header received
0806 22:47:10.196884 (+ 559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:10.196886 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:10.196892 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:10.196940 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:10.198106 (+ 1166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:10.198107 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:10.198108 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:47:10.198112 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:10.198130 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:47:10.198134 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:10.198195 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:10.198290 (+ 95us) server_negotiation.cc:299] Negotiation successful
0806 22:47:10.198307 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 22:47:10.198616 13148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:10.195995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:35555 (local address 127.6.40.65:54329)
0806 22:47:10.196202 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:10.196237 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:10.196752 (+ 515us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:10.197010 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:10.197029 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:10.197037 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:10.197940 (+ 903us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:47:10.197941 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:10.197969 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:47:10.197996 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:47:10.198198 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:47:10.198204 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:47:10.198207 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:47:10.198301 (+ 94us) client_negotiation.cc:240] Negotiation successful
0806 22:47:10.198365 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":99,"threads_started":1}
I20260806 22:47:10.199092 13145 heartbeater.cc:347] Connected to a master server at 127.6.40.126:35555
I20260806 22:47:10.199177 13145 heartbeater.cc:464] Registering TS with master...
I20260806 22:47:10.199332 13145 heartbeater.cc:511] Master 127.6.40.126:35555 requested a full tablet report, sending...
I20260806 22:47:10.199869 12880 ts_manager.cc:194] Registered new tserver with Master: 443abc52a3ab4fbe9b9512fdf6281945 (127.6.40.65:41591)
I20260806 22:47:10.204581 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 12966
I20260806 22:47:10.204675 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056405724609-6305-0/minicluster-data/ts-0/wal/instance
I20260806 22:47:10.204922 6305 external_mini_cluster.cc:1428] Setting key e41c73fd41434e2dbe93a7578680d397
I20260806 22:47:10.205935 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056405724609-6305-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056405724609-6305-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056405724609-6305-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.66:0
--local_ip_for_outbound_sockets=127.6.40.66
--webserver_interface=127.6.40.66
--webserver_port=0
--tserver_master_addrs=127.6.40.126:35555
--builtin_ntp_servers=127.6.40.84:45121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 22:47:10.296000 13151 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:47:10.296240 13151 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:47:10.296267 13151 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:47:10.297560 13151 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:47:10.297609 13151 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:47:10.297638 13151 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.66
I20260806 22:47:10.299103 13151 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:45121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056405724609-6305-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056405724609-6305-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.40.66
--webserver_port=0
--tserver_master_addrs=127.6.40.126:35555
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13151
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.66
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056405724609-6305-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:47:10.299372 13151 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:47:10.299710 13151 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:47:10.305089 13162 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:10.305119 13163 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:10.305089 13165 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:10.305116 13151 server_base.cc:1034] running on GCE node
I20260806 22:47:10.305490 13151 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:47:10.305747 13151 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:47:10.306915 13151 hybrid_clock.cc:648] HybridClock initialized: now 1786056430306906 us; error 32 us; skew 500 ppm
I20260806 22:47:10.308153 13151 webserver.cc:460] Webserver started at http://127.6.40.66:43615/ using document root <none> and password file <none>
I20260806 22:47:10.308400 13151 fs_manager.cc:359] Metadata directory not provided
I20260806 22:47:10.308447 13151 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:47:10.308584 13151 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:47:10.309743 13151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056405724609-6305-0/minicluster-data/ts-1/data/instance:
uuid: "180cfb6f77be446fa9a8b421188f10c9"
format_stamp: "Formatted at 2026-08-06 22:47:10 on dist-test-slave-zh2d"
server_key: "0ae89be6878b89d2a24f25a672a4fa42"
server_key_iv: "5117adaef50591225c8959b8214856ac"
server_key_version: "encryptionkey@0"
I20260806 22:47:10.310112 13151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056405724609-6305-0/minicluster-data/ts-1/wal/instance:
uuid: "180cfb6f77be446fa9a8b421188f10c9"
format_stamp: "Formatted at 2026-08-06 22:47:10 on dist-test-slave-zh2d"
server_key: "0ae89be6878b89d2a24f25a672a4fa42"
server_key_iv: "5117adaef50591225c8959b8214856ac"
server_key_version: "encryptionkey@0"
I20260806 22:47:10.311498 13151 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 22:47:10.312304 13176 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:10.312477 13151 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:47:10.312587 13151 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056405724609-6305-0/minicluster-data/ts-1/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056405724609-6305-0/minicluster-data/ts-1/wal
uuid: "180cfb6f77be446fa9a8b421188f10c9"
format_stamp: "Formatted at 2026-08-06 22:47:10 on dist-test-slave-zh2d"
server_key: "0ae89be6878b89d2a24f25a672a4fa42"
server_key_iv: "5117adaef50591225c8959b8214856ac"
server_key_version: "encryptionkey@0"
I20260806 22:47:10.312665 13151 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056405724609-6305-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056405724609-6305-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056405724609-6305-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:47:10.316068 13151 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:47:10.316347 13151 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:47:10.316493 13151 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:47:10.316718 13151 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:47:10.317065 13151 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:47:10.317119 13151 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:10.317160 13151 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:47:10.317188 13151 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:10.323464 13151 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.66:42441
I20260806 22:47:10.323522 13334 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.66:42441 every 8 connection(s)
I20260806 22:47:10.323879 13151 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
I20260806 22:47:10.326800 12937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:10.324618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.66:37885 (local address 127.6.40.126:35555)
0806 22:47:10.324679 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:10.324681 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:10.324922 (+ 241us) server_negotiation.cc:407] Connection header received
0806 22:47:10.325415 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:10.325416 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:10.325420 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:10.325480 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:10.326628 (+ 1148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:10.326629 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:10.326629 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:47:10.326632 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:10.326645 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:47:10.326649 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:10.326690 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:10.326736 (+ 46us) server_negotiation.cc:299] Negotiation successful
0806 22:47:10.326748 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 22:47:10.327001 13341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:10.324590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:35555 (local address 127.6.40.66:37885)
0806 22:47:10.324795 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:10.324848 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:10.325329 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:10.325530 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:10.325534 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:10.325542 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:10.326501 (+ 959us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:47:10.326502 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:10.326529 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:47:10.326557 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:47:10.326689 (+ 132us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:47:10.326691 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:47:10.326693 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 22:47:10.326741 (+ 48us) client_negotiation.cc:240] Negotiation successful
0806 22:47:10.326786 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":113,"threads_started":1}
I20260806 22:47:10.327453 13338 heartbeater.cc:347] Connected to a master server at 127.6.40.126:35555
I20260806 22:47:10.327548 13338 heartbeater.cc:464] Registering TS with master...
I20260806 22:47:10.327653 13338 heartbeater.cc:511] Master 127.6.40.126:35555 requested a full tablet report, sending...
I20260806 22:47:10.328017 12880 ts_manager.cc:194] Registered new tserver with Master: 180cfb6f77be446fa9a8b421188f10c9 (127.6.40.66:42441)
I20260806 22:47:10.331528 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 13151
I20260806 22:47:10.331641 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056405724609-6305-0/minicluster-data/ts-1/wal/instance
I20260806 22:47:10.331826 6305 external_mini_cluster.cc:1428] Setting key 20c2b1ccada1a3f888650f8c588ed068
I20260806 22:47:10.332763 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056405724609-6305-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056405724609-6305-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056405724609-6305-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.67:0
--local_ip_for_outbound_sockets=127.6.40.67
--webserver_interface=127.6.40.67
--webserver_port=0
--tserver_master_addrs=127.6.40.126:35555
--builtin_ntp_servers=127.6.40.84:45121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 22:47:10.417323 13342 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:47:10.417539 13342 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:47:10.417563 13342 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:47:10.418762 13342 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:47:10.418805 13342 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:47:10.418860 13342 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.67
I20260806 22:47:10.420208 13342 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:45121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056405724609-6305-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056405724609-6305-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.40.67
--webserver_port=0
--tserver_master_addrs=127.6.40.126:35555
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13342
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.67
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056405724609-6305-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:47:10.420469 13342 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:47:10.420787 13342 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:47:10.426090 13347 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:10.426126 13348 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:10.426164 13342 server_base.cc:1034] running on GCE node
W20260806 22:47:10.426132 13351 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:10.426506 13342 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:47:10.426707 13342 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:47:10.427892 13342 hybrid_clock.cc:648] HybridClock initialized: now 1786056430427879 us; error 41 us; skew 500 ppm
I20260806 22:47:10.429193 13342 webserver.cc:460] Webserver started at http://127.6.40.67:36601/ using document root <none> and password file <none>
I20260806 22:47:10.429447 13342 fs_manager.cc:359] Metadata directory not provided
I20260806 22:47:10.429510 13342 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:47:10.429680 13342 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:47:10.430742 13342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056405724609-6305-0/minicluster-data/ts-2/data/instance:
uuid: "0983cc97cb1a42c6bee4d0090a39b438"
format_stamp: "Formatted at 2026-08-06 22:47:10 on dist-test-slave-zh2d"
server_key: "375d1bfddec14fbac35199d1bfe4599e"
server_key_iv: "a24335f75ee3115dbce4a01d2ba108a8"
server_key_version: "encryptionkey@0"
I20260806 22:47:10.431126 13342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056405724609-6305-0/minicluster-data/ts-2/wal/instance:
uuid: "0983cc97cb1a42c6bee4d0090a39b438"
format_stamp: "Formatted at 2026-08-06 22:47:10 on dist-test-slave-zh2d"
server_key: "375d1bfddec14fbac35199d1bfe4599e"
server_key_iv: "a24335f75ee3115dbce4a01d2ba108a8"
server_key_version: "encryptionkey@0"
I20260806 22:47:10.432516 13342 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:47:10.433380 13360 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:10.433578 13342 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:10.433679 13342 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056405724609-6305-0/minicluster-data/ts-2/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056405724609-6305-0/minicluster-data/ts-2/wal
uuid: "0983cc97cb1a42c6bee4d0090a39b438"
format_stamp: "Formatted at 2026-08-06 22:47:10 on dist-test-slave-zh2d"
server_key: "375d1bfddec14fbac35199d1bfe4599e"
server_key_iv: "a24335f75ee3115dbce4a01d2ba108a8"
server_key_version: "encryptionkey@0"
I20260806 22:47:10.433765 13342 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056405724609-6305-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056405724609-6305-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056405724609-6305-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:47:10.437214 13342 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:47:10.437502 13342 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:47:10.437652 13342 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:47:10.437862 13342 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:47:10.438166 13342 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:47:10.438221 13342 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:10.438261 13342 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:47:10.438287 13342 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:10.444690 13342 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.67:45125
I20260806 22:47:10.444741 13519 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.67:45125 every 8 connection(s)
I20260806 22:47:10.445127 13342 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
I20260806 22:47:10.447962 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 13342
I20260806 22:47:10.448086 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056405724609-6305-0/minicluster-data/ts-2/wal/instance
I20260806 22:47:10.448318 6305 external_mini_cluster.cc:1428] Setting key 1d7731d7f4eb6590e97bb3fb95ce73b4
I20260806 22:47:10.448545 12937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:10.445867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.67:52785 (local address 127.6.40.126:35555)
0806 22:47:10.445913 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:10.445915 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:10.446208 (+ 293us) server_negotiation.cc:407] Connection header received
0806 22:47:10.446754 (+ 546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:10.446756 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:10.446766 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:10.446833 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:10.448182 (+ 1349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:10.448185 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:10.448185 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:47:10.448191 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:10.448211 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:47:10.448218 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:10.448255 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:10.448413 (+ 158us) server_negotiation.cc:299] Negotiation successful
0806 22:47:10.448436 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 22:47:10.448920 13524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:10.445874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:35555 (local address 127.6.40.67:52785)
0806 22:47:10.446084 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:10.446118 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:10.446643 (+ 525us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:10.446907 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:10.446916 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:10.446925 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:10.447965 (+ 1040us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:47:10.447965 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:10.448006 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:47:10.448049 (+ 43us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:47:10.448303 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:47:10.448309 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:47:10.448313 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:47:10.448397 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 22:47:10.448467 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":98,"threads_started":1}
I20260806 22:47:10.449546 13520 heartbeater.cc:347] Connected to a master server at 127.6.40.126:35555
I20260806 22:47:10.449656 13520 heartbeater.cc:464] Registering TS with master...
I20260806 22:47:10.449808 13520 heartbeater.cc:511] Master 127.6.40.126:35555 requested a full tablet report, sending...
I20260806 22:47:10.450213 12880 ts_manager.cc:194] Registered new tserver with Master: 0983cc97cb1a42c6bee4d0090a39b438 (127.6.40.67:45125)
I20260806 22:47:10.460376 6305 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:47:10.463120 12937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:10.462083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50544 (local address 127.6.40.126:35555)
0806 22:47:10.462163 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:10.462165 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:10.462240 (+ 75us) server_negotiation.cc:407] Connection header received
0806 22:47:10.462296 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:10.462296 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:10.462302 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:10.462337 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:10.462890 (+ 553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:10.462891 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:10.462891 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:47:10.462895 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:10.462910 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:47:10.462915 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:10.462968 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:10.463001 (+ 33us) server_negotiation.cc:299] Negotiation successful
0806 22:47:10.463013 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 22:47:10.464829 12880 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50544:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:47:10.466444 12880 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:47:10.470666 13538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:10.469003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.66:42441 (local address 127.0.0.1:36834)
0806 22:47:10.469150 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:10.469159 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:10.469201 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:10.469652 (+ 451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:10.469660 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:10.469666 (+ 6us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:10.470185 (+ 519us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:47:10.470185 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:10.470199 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:47:10.470219 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:47:10.470484 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:47:10.470491 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:47:10.470494 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:47:10.470558 (+ 64us) client_negotiation.cc:240] Negotiation successful
0806 22:47:10.470571 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":101,"threads_started":1}
I20260806 22:47:10.470984 13539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:10.469271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36834 (local address 127.6.40.66:42441)
0806 22:47:10.469513 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:10.469518 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:10.469528 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:47:10.469559 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:10.469561 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:10.469572 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:10.469654 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:10.470339 (+ 685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:10.470348 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:10.470348 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:47:10.470353 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:10.470394 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:47:10.470430 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:10.470490 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:10.470864 (+ 374us) server_negotiation.cc:299] Negotiation successful
0806 22:47:10.470884 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":94,"threads_started":1}
I20260806 22:47:10.470996 13534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:10.468733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.67:45125 (local address 127.0.0.1:56788)
0806 22:47:10.468962 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:10.469008 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:10.469073 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:10.469875 (+ 802us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:10.469876 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:10.469879 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:10.470593 (+ 714us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:47:10.470594 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:10.470601 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:47:10.470616 (+ 15us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:47:10.470870 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:47:10.470870 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:47:10.470871 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 22:47:10.470913 (+ 42us) client_negotiation.cc:240] Negotiation successful
0806 22:47:10.470926 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":59,"threads_started":1}
I20260806 22:47:10.471101 13532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:10.468750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.65:41591 (local address 127.0.0.1:46892)
0806 22:47:10.469135 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:10.469148 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:10.469202 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:10.470275 (+ 1073us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:10.470276 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:10.470279 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:10.470674 (+ 395us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:47:10.470675 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:10.470681 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:47:10.470694 (+ 13us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:47:10.470951 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:47:10.470963 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:47:10.470964 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 22:47:10.471012 (+ 48us) client_negotiation.cc:240] Negotiation successful
0806 22:47:10.471026 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":142,"threads_started":1}
I20260806 22:47:10.471287 13536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:10.468927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56788 (local address 127.6.40.67:45125)
0806 22:47:10.469335 (+ 408us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:10.469341 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:10.469352 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:47:10.469404 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:10.469406 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:10.469417 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:10.469497 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:10.470695 (+ 1198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:10.470703 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:10.470703 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:47:10.470708 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:10.470750 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:47:10.470779 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:10.470843 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:10.471177 (+ 334us) server_negotiation.cc:299] Negotiation successful
0806 22:47:10.471199 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":103,"threads_started":1}
I20260806 22:47:10.471709 13245 tablet_service.cc:1467] Processing CreateTablet for tablet 0187249123a84f0d8438700c96fec070 (DEFAULT_TABLE table=test-table [id=7c7d139f99054145a1da416340122f8c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:47:10.471961 13431 tablet_service.cc:1467] Processing CreateTablet for tablet 0187249123a84f0d8438700c96fec070 (DEFAULT_TABLE table=test-table [id=7c7d139f99054145a1da416340122f8c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:47:10.472078 13245 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0187249123a84f0d8438700c96fec070. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:47:10.472313 13431 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0187249123a84f0d8438700c96fec070. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:47:10.474833 13540 tablet_bootstrap.cc:492] T 0187249123a84f0d8438700c96fec070 P 180cfb6f77be446fa9a8b421188f10c9: Bootstrap starting.
I20260806 22:47:10.474885 13541 tablet_bootstrap.cc:492] T 0187249123a84f0d8438700c96fec070 P 0983cc97cb1a42c6bee4d0090a39b438: Bootstrap starting.
I20260806 22:47:10.475220 13535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:10.468892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46892 (local address 127.6.40.65:41591)
0806 22:47:10.470118 (+ 1226us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:10.470123 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:10.470133 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:47:10.470163 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:10.470165 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:10.470174 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:10.470257 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:10.470786 (+ 529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:10.470793 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:10.470794 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:47:10.470798 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:10.470840 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:47:10.470873 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:10.470926 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:10.475071 (+ 4145us) server_negotiation.cc:299] Negotiation successful
0806 22:47:10.475103 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1178,"thread_start_us":111,"threads_started":1}
I20260806 22:47:10.475579 13540 tablet_bootstrap.cc:654] T 0187249123a84f0d8438700c96fec070 P 180cfb6f77be446fa9a8b421188f10c9: Neither blocks nor log segments found. Creating new log.
I20260806 22:47:10.475878 13541 tablet_bootstrap.cc:654] T 0187249123a84f0d8438700c96fec070 P 0983cc97cb1a42c6bee4d0090a39b438: Neither blocks nor log segments found. Creating new log.
I20260806 22:47:10.475932 13540 log.cc:824] T 0187249123a84f0d8438700c96fec070 P 180cfb6f77be446fa9a8b421188f10c9: Log is configured to *not* fsync() on all Append() calls
I20260806 22:47:10.475896 13059 tablet_service.cc:1467] Processing CreateTablet for tablet 0187249123a84f0d8438700c96fec070 (DEFAULT_TABLE table=test-table [id=7c7d139f99054145a1da416340122f8c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:47:10.476243 13541 log.cc:824] T 0187249123a84f0d8438700c96fec070 P 0983cc97cb1a42c6bee4d0090a39b438: Log is configured to *not* fsync() on all Append() calls
I20260806 22:47:10.476275 13059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0187249123a84f0d8438700c96fec070. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:47:10.476764 13540 tablet_bootstrap.cc:492] T 0187249123a84f0d8438700c96fec070 P 180cfb6f77be446fa9a8b421188f10c9: No bootstrap required, opened a new log
I20260806 22:47:10.476871 13540 ts_tablet_manager.cc:1397] T 0187249123a84f0d8438700c96fec070 P 180cfb6f77be446fa9a8b421188f10c9: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:47:10.477027 13541 tablet_bootstrap.cc:492] T 0187249123a84f0d8438700c96fec070 P 0983cc97cb1a42c6bee4d0090a39b438: No bootstrap required, opened a new log
I20260806 22:47:10.477113 13541 ts_tablet_manager.cc:1397] T 0187249123a84f0d8438700c96fec070 P 0983cc97cb1a42c6bee4d0090a39b438: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:47:10.479179 13541 raft_consensus.cc:348] T 0187249123a84f0d8438700c96fec070 P 0983cc97cb1a42c6bee4d0090a39b438 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0983cc97cb1a42c6bee4d0090a39b438" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45125 } } peers { permanent_uuid: "443abc52a3ab4fbe9b9512fdf6281945" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 41591 } } peers { permanent_uuid: "180cfb6f77be446fa9a8b421188f10c9" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 42441 } }
I20260806 22:47:10.479337 13541 raft_consensus.cc:374] T 0187249123a84f0d8438700c96fec070 P 0983cc97cb1a42c6bee4d0090a39b438 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:47:10.479383 13541 raft_consensus.cc:729] T 0187249123a84f0d8438700c96fec070 P 0983cc97cb1a42c6bee4d0090a39b438 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0983cc97cb1a42c6bee4d0090a39b438, State: Initialized, Role: FOLLOWER
I20260806 22:47:10.479511 13541 consensus_queue.cc:260] T 0187249123a84f0d8438700c96fec070 P 0983cc97cb1a42c6bee4d0090a39b438 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0983cc97cb1a42c6bee4d0090a39b438" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45125 } } peers { permanent_uuid: "443abc52a3ab4fbe9b9512fdf6281945" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 41591 } } peers { permanent_uuid: "180cfb6f77be446fa9a8b421188f10c9" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 42441 } }
I20260806 22:47:10.479606 13540 raft_consensus.cc:348] T 0187249123a84f0d8438700c96fec070 P 180cfb6f77be446fa9a8b421188f10c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0983cc97cb1a42c6bee4d0090a39b438" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45125 } } peers { permanent_uuid: "443abc52a3ab4fbe9b9512fdf6281945" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 41591 } } peers { permanent_uuid: "180cfb6f77be446fa9a8b421188f10c9" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 42441 } }
I20260806 22:47:10.479771 13540 raft_consensus.cc:374] T 0187249123a84f0d8438700c96fec070 P 180cfb6f77be446fa9a8b421188f10c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:47:10.479813 13541 ts_tablet_manager.cc:1428] T 0187249123a84f0d8438700c96fec070 P 0983cc97cb1a42c6bee4d0090a39b438: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:47:10.479822 13540 raft_consensus.cc:729] T 0187249123a84f0d8438700c96fec070 P 180cfb6f77be446fa9a8b421188f10c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 180cfb6f77be446fa9a8b421188f10c9, State: Initialized, Role: FOLLOWER
I20260806 22:47:10.479936 13520 heartbeater.cc:503] Master 127.6.40.126:35555 was elected leader, sending a full tablet report...
I20260806 22:47:10.479934 13540 consensus_queue.cc:260] T 0187249123a84f0d8438700c96fec070 P 180cfb6f77be446fa9a8b421188f10c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0983cc97cb1a42c6bee4d0090a39b438" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45125 } } peers { permanent_uuid: "443abc52a3ab4fbe9b9512fdf6281945" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 41591 } } peers { permanent_uuid: "180cfb6f77be446fa9a8b421188f10c9" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 42441 } }
I20260806 22:47:10.480201 13540 ts_tablet_manager.cc:1428] T 0187249123a84f0d8438700c96fec070 P 180cfb6f77be446fa9a8b421188f10c9: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:47:10.480425 13544 tablet_bootstrap.cc:492] T 0187249123a84f0d8438700c96fec070 P 443abc52a3ab4fbe9b9512fdf6281945: Bootstrap starting.
I20260806 22:47:10.480645 13338 heartbeater.cc:503] Master 127.6.40.126:35555 was elected leader, sending a full tablet report...
I20260806 22:47:10.481269 13544 tablet_bootstrap.cc:654] T 0187249123a84f0d8438700c96fec070 P 443abc52a3ab4fbe9b9512fdf6281945: Neither blocks nor log segments found. Creating new log.
I20260806 22:47:10.481639 13544 log.cc:824] T 0187249123a84f0d8438700c96fec070 P 443abc52a3ab4fbe9b9512fdf6281945: Log is configured to *not* fsync() on all Append() calls
I20260806 22:47:10.482385 13544 tablet_bootstrap.cc:492] T 0187249123a84f0d8438700c96fec070 P 443abc52a3ab4fbe9b9512fdf6281945: No bootstrap required, opened a new log
I20260806 22:47:10.482517 13544 ts_tablet_manager.cc:1397] T 0187249123a84f0d8438700c96fec070 P 443abc52a3ab4fbe9b9512fdf6281945: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260806 22:47:10.484488 13544 raft_consensus.cc:348] T 0187249123a84f0d8438700c96fec070 P 443abc52a3ab4fbe9b9512fdf6281945 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0983cc97cb1a42c6bee4d0090a39b438" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45125 } } peers { permanent_uuid: "443abc52a3ab4fbe9b9512fdf6281945" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 41591 } } peers { permanent_uuid: "180cfb6f77be446fa9a8b421188f10c9" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 42441 } }
I20260806 22:47:10.484638 13544 raft_consensus.cc:374] T 0187249123a84f0d8438700c96fec070 P 443abc52a3ab4fbe9b9512fdf6281945 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:47:10.484709 13544 raft_consensus.cc:729] T 0187249123a84f0d8438700c96fec070 P 443abc52a3ab4fbe9b9512fdf6281945 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 443abc52a3ab4fbe9b9512fdf6281945, State: Initialized, Role: FOLLOWER
I20260806 22:47:10.484830 13544 consensus_queue.cc:260] T 0187249123a84f0d8438700c96fec070 P 443abc52a3ab4fbe9b9512fdf6281945 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0983cc97cb1a42c6bee4d0090a39b438" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45125 } } peers { permanent_uuid: "443abc52a3ab4fbe9b9512fdf6281945" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 41591 } } peers { permanent_uuid: "180cfb6f77be446fa9a8b421188f10c9" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 42441 } }
I20260806 22:47:10.485137 13544 ts_tablet_manager.cc:1428] T 0187249123a84f0d8438700c96fec070 P 443abc52a3ab4fbe9b9512fdf6281945: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260806 22:47:10.485198 13145 heartbeater.cc:503] Master 127.6.40.126:35555 was elected leader, sending a full tablet report...
W20260806 22:47:10.574455 13339 tablet.cc:2369] T 0187249123a84f0d8438700c96fec070 P 180cfb6f77be446fa9a8b421188f10c9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:47:10.634743 13545 raft_consensus.cc:482] T 0187249123a84f0d8438700c96fec070 P 0983cc97cb1a42c6bee4d0090a39b438 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:47:10.634877 13545 raft_consensus.cc:504] T 0187249123a84f0d8438700c96fec070 P 0983cc97cb1a42c6bee4d0090a39b438 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0983cc97cb1a42c6bee4d0090a39b438" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45125 } } peers { permanent_uuid: "443abc52a3ab4fbe9b9512fdf6281945" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 41591 } } peers { permanent_uuid: "180cfb6f77be446fa9a8b421188f10c9" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 42441 } }
I20260806 22:47:10.635337 13545 leader_election.cc:288] T 0187249123a84f0d8438700c96fec070 P 0983cc97cb1a42c6bee4d0090a39b438 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 443abc52a3ab4fbe9b9512fdf6281945 (127.6.40.65:41591), 180cfb6f77be446fa9a8b421188f10c9 (127.6.40.66:42441)
I20260806 22:47:10.636885 13524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:10.635562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.65:41591 (local address 127.6.40.67:36865)
0806 22:47:10.635641 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:10.635657 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:10.635723 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:10.635995 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:10.635996 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:10.635999 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:10.636524 (+ 525us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:47:10.636525 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:10.636534 (+ 9us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:47:10.636549 (+ 15us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:47:10.636723 (+ 174us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:47:10.636724 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:47:10.636724 (+ 0us) client_negotiation.cc:769] Sending connection context
0806 22:47:10.636770 (+ 46us) client_negotiation.cc:240] Negotiation successful
0806 22:47:10.636785 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260806 22:47:10.636885 13535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:10.635665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.67:36865 (local address 127.6.40.65:41591)
0806 22:47:10.635746 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:10.635748 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:10.635773 (+ 25us) server_negotiation.cc:407] Connection header received
0806 22:47:10.635803 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:10.635804 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:10.635811 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:10.635973 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:10.636641 (+ 668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:10.636642 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:10.636642 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:47:10.636646 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:10.636662 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:47:10.636666 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:10.636724 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:10.636770 (+ 46us) server_negotiation.cc:299] Negotiation successful
0806 22:47:10.636785 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 22:47:10.637117 13539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:10.635813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.67:60963 (local address 127.6.40.66:42441)
0806 22:47:10.635879 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:10.635881 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:10.636010 (+ 129us) server_negotiation.cc:407] Connection header received
0806 22:47:10.636051 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:10.636052 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:10.636059 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:10.636116 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:10.636831 (+ 715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:10.636855 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:10.636855 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:47:10.636861 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:10.636878 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:47:10.636884 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:10.636936 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:10.637037 (+ 101us) server_negotiation.cc:299] Negotiation successful
0806 22:47:10.637053 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 22:47:10.637153 13550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:10.635716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.66:42441 (local address 127.6.40.67:60963)
0806 22:47:10.635906 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:10.635917 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:10.635967 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:10.636126 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:10.636127 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:10.636129 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:10.636712 (+ 583us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:47:10.636712 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:10.636719 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:47:10.636731 (+ 12us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:47:10.636973 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:47:10.636974 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:47:10.636974 (+ 0us) client_negotiation.cc:769] Sending connection context
0806 22:47:10.637033 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 22:47:10.637049 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":108,"threads_started":1}
I20260806 22:47:10.637365 13089 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0187249123a84f0d8438700c96fec070" candidate_uuid: "0983cc97cb1a42c6bee4d0090a39b438" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "443abc52a3ab4fbe9b9512fdf6281945" is_pre_election: true
I20260806 22:47:10.637501 13089 raft_consensus.cc:2454] T 0187249123a84f0d8438700c96fec070 P 443abc52a3ab4fbe9b9512fdf6281945 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0983cc97cb1a42c6bee4d0090a39b438 in term 0.
I20260806 22:47:10.637794 13362 leader_election.cc:302] T 0187249123a84f0d8438700c96fec070 P 0983cc97cb1a42c6bee4d0090a39b438 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0983cc97cb1a42c6bee4d0090a39b438, 443abc52a3ab4fbe9b9512fdf6281945; no voters:
I20260806 22:47:10.637959 13545 raft_consensus.cc:2781] T 0187249123a84f0d8438700c96fec070 P 0983cc97cb1a42c6bee4d0090a39b438 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:47:10.638052 13545 raft_consensus.cc:482] T 0187249123a84f0d8438700c96fec070 P 0983cc97cb1a42c6bee4d0090a39b438 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:47:10.638085 13545 raft_consensus.cc:3037] T 0187249123a84f0d8438700c96fec070 P 0983cc97cb1a42c6bee4d0090a39b438 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:47:10.638237 13269 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0187249123a84f0d8438700c96fec070" candidate_uuid: "0983cc97cb1a42c6bee4d0090a39b438" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "180cfb6f77be446fa9a8b421188f10c9" is_pre_election: true
I20260806 22:47:10.638410 13269 raft_consensus.cc:2454] T 0187249123a84f0d8438700c96fec070 P 180cfb6f77be446fa9a8b421188f10c9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0983cc97cb1a42c6bee4d0090a39b438 in term 0.
I20260806 22:47:10.639078 13545 raft_consensus.cc:504] T 0187249123a84f0d8438700c96fec070 P 0983cc97cb1a42c6bee4d0090a39b438 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0983cc97cb1a42c6bee4d0090a39b438" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45125 } } peers { permanent_uuid: "443abc52a3ab4fbe9b9512fdf6281945" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 41591 } } peers { permanent_uuid: "180cfb6f77be446fa9a8b421188f10c9" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 42441 } }
I20260806 22:47:10.639194 13545 leader_election.cc:288] T 0187249123a84f0d8438700c96fec070 P 0983cc97cb1a42c6bee4d0090a39b438 [CANDIDATE]: Term 1 election: Requested vote from peers 443abc52a3ab4fbe9b9512fdf6281945 (127.6.40.65:41591), 180cfb6f77be446fa9a8b421188f10c9 (127.6.40.66:42441)
I20260806 22:47:10.639444 13269 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0187249123a84f0d8438700c96fec070" candidate_uuid: "0983cc97cb1a42c6bee4d0090a39b438" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "180cfb6f77be446fa9a8b421188f10c9"
I20260806 22:47:10.639446 13089 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0187249123a84f0d8438700c96fec070" candidate_uuid: "0983cc97cb1a42c6bee4d0090a39b438" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "443abc52a3ab4fbe9b9512fdf6281945"
I20260806 22:47:10.639554 13269 raft_consensus.cc:3037] T 0187249123a84f0d8438700c96fec070 P 180cfb6f77be446fa9a8b421188f10c9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:47:10.639554 13089 raft_consensus.cc:3037] T 0187249123a84f0d8438700c96fec070 P 443abc52a3ab4fbe9b9512fdf6281945 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:47:10.640632 13269 raft_consensus.cc:2454] T 0187249123a84f0d8438700c96fec070 P 180cfb6f77be446fa9a8b421188f10c9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0983cc97cb1a42c6bee4d0090a39b438 in term 1.
I20260806 22:47:10.640632 13089 raft_consensus.cc:2454] T 0187249123a84f0d8438700c96fec070 P 443abc52a3ab4fbe9b9512fdf6281945 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0983cc97cb1a42c6bee4d0090a39b438 in term 1.
I20260806 22:47:10.640914 13362 leader_election.cc:302] T 0187249123a84f0d8438700c96fec070 P 0983cc97cb1a42c6bee4d0090a39b438 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0983cc97cb1a42c6bee4d0090a39b438, 180cfb6f77be446fa9a8b421188f10c9; no voters:
I20260806 22:47:10.641037 13545 raft_consensus.cc:2781] T 0187249123a84f0d8438700c96fec070 P 0983cc97cb1a42c6bee4d0090a39b438 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:47:10.641206 13545 raft_consensus.cc:686] T 0187249123a84f0d8438700c96fec070 P 0983cc97cb1a42c6bee4d0090a39b438 [term 1 LEADER]: Becoming Leader. State: Replica: 0983cc97cb1a42c6bee4d0090a39b438, State: Running, Role: LEADER
I20260806 22:47:10.641304 13545 consensus_queue.cc:237] T 0187249123a84f0d8438700c96fec070 P 0983cc97cb1a42c6bee4d0090a39b438 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0983cc97cb1a42c6bee4d0090a39b438" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45125 } } peers { permanent_uuid: "443abc52a3ab4fbe9b9512fdf6281945" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 41591 } } peers { permanent_uuid: "180cfb6f77be446fa9a8b421188f10c9" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 42441 } }
I20260806 22:47:10.642283 12879 catalog_manager.cc:5495] T 0187249123a84f0d8438700c96fec070 P 0983cc97cb1a42c6bee4d0090a39b438 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0983cc97cb1a42c6bee4d0090a39b438 (127.6.40.67). New cstate: current_term: 1 leader_uuid: "0983cc97cb1a42c6bee4d0090a39b438" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0983cc97cb1a42c6bee4d0090a39b438" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45125 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "443abc52a3ab4fbe9b9512fdf6281945" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 41591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "180cfb6f77be446fa9a8b421188f10c9" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 42441 } health_report { overall_health: UNKNOWN } } }
I20260806 22:47:10.655699 13536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:10.654755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56804 (local address 127.6.40.67:45125)
0806 22:47:10.654814 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:10.654816 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:10.654824 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:47:10.654879 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:10.654880 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:10.654892 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:10.654940 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:10.655469 (+ 529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:10.655470 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:10.655471 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:47:10.655474 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:10.655489 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:47:10.655493 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:10.655539 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:10.655586 (+ 47us) server_negotiation.cc:299] Negotiation successful
0806 22:47:10.655597 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 22:47:10.657671 13089 raft_consensus.cc:1261] T 0187249123a84f0d8438700c96fec070 P 443abc52a3ab4fbe9b9512fdf6281945 [term 1 FOLLOWER]: Refusing update from remote peer 0983cc97cb1a42c6bee4d0090a39b438: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:47:10.657671 13269 raft_consensus.cc:1261] T 0187249123a84f0d8438700c96fec070 P 180cfb6f77be446fa9a8b421188f10c9 [term 1 FOLLOWER]: Refusing update from remote peer 0983cc97cb1a42c6bee4d0090a39b438: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:47:10.658064 13552 consensus_queue.cc:1035] T 0187249123a84f0d8438700c96fec070 P 0983cc97cb1a42c6bee4d0090a39b438 [LEADER]: Connected to new peer: Peer: permanent_uuid: "443abc52a3ab4fbe9b9512fdf6281945" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 41591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:47:10.658206 13545 consensus_queue.cc:1035] T 0187249123a84f0d8438700c96fec070 P 0983cc97cb1a42c6bee4d0090a39b438 [LEADER]: Connected to new peer: Peer: permanent_uuid: "180cfb6f77be446fa9a8b421188f10c9" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 42441 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:47:10.661355 13558 mvcc.cc:205] Tried to move back new op lower bound from 7315687139970809856 to 7315687139907543040. Current Snapshot: MvccSnapshot[applied={T|T < 7315687139970809856}]
I20260806 22:47:10.662963 12880 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:50544:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:47:10.663085 12880 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:50544:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:47:10.663883 12880 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 073e1a46d5b9473380522d27c66148df: Sending DeleteTablet for 3 replicas of tablet 0187249123a84f0d8438700c96fec070
I20260806 22:47:10.664199 13431 tablet_service.cc:1514] Processing DeleteTablet for tablet 0187249123a84f0d8438700c96fec070 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:47:10 UTC) from {username='slave'} at 127.0.0.1:56788
I20260806 22:47:10.664294 13245 tablet_service.cc:1514] Processing DeleteTablet for tablet 0187249123a84f0d8438700c96fec070 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:47:10 UTC) from {username='slave'} at 127.0.0.1:36834
I20260806 22:47:10.664294 13059 tablet_service.cc:1514] Processing DeleteTablet for tablet 0187249123a84f0d8438700c96fec070 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:47:10 UTC) from {username='slave'} at 127.0.0.1:46892
I20260806 22:47:10.664557 13570 tablet_replica.cc:330] T 0187249123a84f0d8438700c96fec070 P 180cfb6f77be446fa9a8b421188f10c9: stopping tablet replica
I20260806 22:47:10.664598 13567 tablet_replica.cc:330] T 0187249123a84f0d8438700c96fec070 P 0983cc97cb1a42c6bee4d0090a39b438: stopping tablet replica
I20260806 22:47:10.664667 13570 raft_consensus.cc:2229] T 0187249123a84f0d8438700c96fec070 P 180cfb6f77be446fa9a8b421188f10c9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:47:10.664697 13567 raft_consensus.cc:2229] T 0187249123a84f0d8438700c96fec070 P 0983cc97cb1a42c6bee4d0090a39b438 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:47:10.664769 13570 raft_consensus.cc:2258] T 0187249123a84f0d8438700c96fec070 P 180cfb6f77be446fa9a8b421188f10c9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:47:10.664798 13567 raft_consensus.cc:2258] T 0187249123a84f0d8438700c96fec070 P 0983cc97cb1a42c6bee4d0090a39b438 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:47:10.665058 13569 tablet_replica.cc:330] T 0187249123a84f0d8438700c96fec070 P 443abc52a3ab4fbe9b9512fdf6281945: stopping tablet replica
I20260806 22:47:10.665086 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 12966
I20260806 22:47:10.665158 13569 raft_consensus.cc:2229] T 0187249123a84f0d8438700c96fec070 P 443abc52a3ab4fbe9b9512fdf6281945 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:47:10.665241 13569 raft_consensus.cc:2258] T 0187249123a84f0d8438700c96fec070 P 443abc52a3ab4fbe9b9512fdf6281945 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:47:10.665477 13567 ts_tablet_manager.cc:1905] T 0187249123a84f0d8438700c96fec070 P 0983cc97cb1a42c6bee4d0090a39b438: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:47:10.667917 13570 ts_tablet_manager.cc:1905] T 0187249123a84f0d8438700c96fec070 P 180cfb6f77be446fa9a8b421188f10c9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:47:10.667943 13567 ts_tablet_manager.cc:1918] T 0187249123a84f0d8438700c96fec070 P 0983cc97cb1a42c6bee4d0090a39b438: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:47:10.668012 13567 log.cc:1196] T 0187249123a84f0d8438700c96fec070 P 0983cc97cb1a42c6bee4d0090a39b438: Deleting WAL directory at /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056405724609-6305-0/minicluster-data/ts-2/wal/wals/0187249123a84f0d8438700c96fec070
I20260806 22:47:10.668294 13567 ts_tablet_manager.cc:1939] T 0187249123a84f0d8438700c96fec070 P 0983cc97cb1a42c6bee4d0090a39b438: Deleting consensus metadata
I20260806 22:47:10.670114 13570 ts_tablet_manager.cc:1918] T 0187249123a84f0d8438700c96fec070 P 180cfb6f77be446fa9a8b421188f10c9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:47:10.670118 12859 catalog_manager.cc:4841] TS 0983cc97cb1a42c6bee4d0090a39b438 (127.6.40.67:45125): tablet 0187249123a84f0d8438700c96fec070 (table test-table [id=7c7d139f99054145a1da416340122f8c]) successfully deleted
I20260806 22:47:10.670185 13570 log.cc:1196] T 0187249123a84f0d8438700c96fec070 P 180cfb6f77be446fa9a8b421188f10c9: Deleting WAL directory at /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056405724609-6305-0/minicluster-data/ts-1/wal/wals/0187249123a84f0d8438700c96fec070
I20260806 22:47:10.670444 13570 ts_tablet_manager.cc:1939] T 0187249123a84f0d8438700c96fec070 P 180cfb6f77be446fa9a8b421188f10c9: Deleting consensus metadata
I20260806 22:47:10.670858 12857 catalog_manager.cc:4841] TS 180cfb6f77be446fa9a8b421188f10c9 (127.6.40.66:42441): tablet 0187249123a84f0d8438700c96fec070 (table test-table [id=7c7d139f99054145a1da416340122f8c]) successfully deleted
W20260806 22:47:10.675678 12857 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.40.65:41591 (error 108)
I20260806 22:47:10.676383 13534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:10.676163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.65:41591 (local address 127.0.0.1:46896)
0806 22:47:10.676264 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:10.676336 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.65:41591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260806 22:47:10.676463 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 13151
W20260806 22:47:10.686331 12857 catalog_manager.cc:4584] TS 443abc52a3ab4fbe9b9512fdf6281945 (127.6.40.65:41591): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0187249123a84f0d8438700c96fec070: Network error: Client connection negotiation failed: client connection to 127.6.40.65:41591: connect: Connection refused (error 111)
I20260806 22:47:10.691037 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 13342
I20260806 22:47:10.699227 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 12832
2026-08-06T22:47:10Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (792 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260806 22:47:10.725497 6305 mini_oidc.cc:152] Starting JWKS server
I20260806 22:47:10.726765 6305 webserver.cc:460] Webserver started at https://127.0.0.1:45821/ using document root <none> and password file <none>
I20260806 22:47:10.726895 6305 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 22:47:10.727097 6305 webserver.cc:460] Webserver started at http://127.0.0.1:38499/ using document root <none> and password file <none>
2026-08-06T22:47:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:47:10Z Disabled control of system clock
I20260806 22:47:10.735810 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786056405724609-6305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786056405724609-6305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786056405724609-6305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.126:37373
--webserver_interface=127.6.40.126
--webserver_port=0
--builtin_ntp_servers=127.6.40.84:44779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.40.126:37373
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45821/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786056405724609-6305-0/testchainca.cert with env {}
W20260806 22:47:10.816923 13581 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:47:10.817154 13581 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:47:10.817178 13581 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:47:10.818406 13581 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:47:10.818449 13581 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:47:10.818462 13581 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:47:10.818475 13581 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:47:10.818533 13581 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 22:47:10.818550 13581 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 22:47:10.818572 13581 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:45821/jwks.json
I20260806 22:47:10.820128 13581 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:44779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786056405724609-6305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786056405724609-6305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.40.126:37373
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.126:37373
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45821/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.40.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786056405724609-6305-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13581
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786056405724609-6305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:47:10.820420 13581 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:47:10.820674 13581 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:47:10.825812 13590 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:10.825922 13581 server_base.cc:1034] running on GCE node
W20260806 22:47:10.826012 13593 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:10.825812 13591 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:10.826402 13581 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:47:10.826742 13581 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:47:10.827896 13581 hybrid_clock.cc:648] HybridClock initialized: now 1786056430827884 us; error 29 us; skew 500 ppm
I20260806 22:47:10.829219 13581 webserver.cc:460] Webserver started at http://127.6.40.126:34097/ using document root <none> and password file <none>
I20260806 22:47:10.829473 13581 fs_manager.cc:359] Metadata directory not provided
I20260806 22:47:10.829526 13581 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:47:10.829689 13581 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:47:10.831215 13581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786056405724609-6305-0/minicluster-data/master-0/data/instance:
uuid: "1715ac52324c460babc67288966e9a05"
format_stamp: "Formatted at 2026-08-06 22:47:10 on dist-test-slave-zh2d"
server_key: "50643eb9f1e8ea7b47d089eff4551244"
server_key_iv: "3acaca17d0274f8dc1354ccea88a3209"
server_key_version: "encryptionkey@0"
I20260806 22:47:10.831740 13581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786056405724609-6305-0/minicluster-data/master-0/wal/instance:
uuid: "1715ac52324c460babc67288966e9a05"
format_stamp: "Formatted at 2026-08-06 22:47:10 on dist-test-slave-zh2d"
server_key: "50643eb9f1e8ea7b47d089eff4551244"
server_key_iv: "3acaca17d0274f8dc1354ccea88a3209"
server_key_version: "encryptionkey@0"
I20260806 22:47:10.833220 13581 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.003s
I20260806 22:47:10.833931 13604 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:10.834095 13581 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:47:10.834204 13581 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786056405724609-6305-0/minicluster-data/master-0/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786056405724609-6305-0/minicluster-data/master-0/wal
uuid: "1715ac52324c460babc67288966e9a05"
format_stamp: "Formatted at 2026-08-06 22:47:10 on dist-test-slave-zh2d"
server_key: "50643eb9f1e8ea7b47d089eff4551244"
server_key_iv: "3acaca17d0274f8dc1354ccea88a3209"
server_key_version: "encryptionkey@0"
I20260806 22:47:10.834280 13581 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786056405724609-6305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786056405724609-6305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786056405724609-6305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:47:10.845610 13581 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:47:10.845907 13581 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:47:10.846073 13581 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:47:10.850456 13581 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.126:37373
I20260806 22:47:10.850482 13677 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.126:37373 every 8 connection(s)
I20260806 22:47:10.850824 13581 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
I20260806 22:47:10.851809 13678 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:47:10.854176 13678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1715ac52324c460babc67288966e9a05: Bootstrap starting.
I20260806 22:47:10.854729 13678 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1715ac52324c460babc67288966e9a05: Neither blocks nor log segments found. Creating new log.
I20260806 22:47:10.854986 13678 log.cc:824] T 00000000000000000000000000000000 P 1715ac52324c460babc67288966e9a05: Log is configured to *not* fsync() on all Append() calls
I20260806 22:47:10.855625 13678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1715ac52324c460babc67288966e9a05: No bootstrap required, opened a new log
I20260806 22:47:10.857251 13678 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1715ac52324c460babc67288966e9a05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1715ac52324c460babc67288966e9a05" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 37373 } }
I20260806 22:47:10.857384 13678 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1715ac52324c460babc67288966e9a05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:47:10.857406 13678 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1715ac52324c460babc67288966e9a05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1715ac52324c460babc67288966e9a05, State: Initialized, Role: FOLLOWER
I20260806 22:47:10.857550 13678 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1715ac52324c460babc67288966e9a05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1715ac52324c460babc67288966e9a05" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 37373 } }
I20260806 22:47:10.857630 13678 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1715ac52324c460babc67288966e9a05 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:47:10.857651 13678 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1715ac52324c460babc67288966e9a05 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:47:10.857681 13678 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1715ac52324c460babc67288966e9a05 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:47:10.858144 13678 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1715ac52324c460babc67288966e9a05 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1715ac52324c460babc67288966e9a05" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 37373 } }
I20260806 22:47:10.858228 13678 leader_election.cc:302] T 00000000000000000000000000000000 P 1715ac52324c460babc67288966e9a05 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1715ac52324c460babc67288966e9a05; no voters:
I20260806 22:47:10.858353 13678 leader_election.cc:288] T 00000000000000000000000000000000 P 1715ac52324c460babc67288966e9a05 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:47:10.858430 13682 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1715ac52324c460babc67288966e9a05 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:47:10.858615 13682 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1715ac52324c460babc67288966e9a05 [term 1 LEADER]: Becoming Leader. State: Replica: 1715ac52324c460babc67288966e9a05, State: Running, Role: LEADER
I20260806 22:47:10.858639 13678 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1715ac52324c460babc67288966e9a05 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:47:10.858726 13682 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1715ac52324c460babc67288966e9a05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1715ac52324c460babc67288966e9a05" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 37373 } }
I20260806 22:47:10.859144 13687 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1715ac52324c460babc67288966e9a05 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1715ac52324c460babc67288966e9a05. Latest consensus state: current_term: 1 leader_uuid: "1715ac52324c460babc67288966e9a05" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1715ac52324c460babc67288966e9a05" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 37373 } } }
I20260806 22:47:10.859248 13687 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1715ac52324c460babc67288966e9a05 [sys.catalog]: This master's current role is: LEADER
I20260806 22:47:10.859225 13684 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1715ac52324c460babc67288966e9a05 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1715ac52324c460babc67288966e9a05" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1715ac52324c460babc67288966e9a05" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 37373 } } }
I20260806 22:47:10.859666 13684 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1715ac52324c460babc67288966e9a05 [sys.catalog]: This master's current role is: LEADER
I20260806 22:47:10.859727 13696 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:47:10.860242 13696 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:47:10.860667 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 13581
I20260806 22:47:10.860782 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786056405724609-6305-0/minicluster-data/master-0/wal/instance
I20260806 22:47:10.861008 6305 external_mini_cluster.cc:1428] Setting key 7a4e1493dbc2c0516dfaa3c5de7f386e
I20260806 22:47:10.861896 13696 catalog_manager.cc:1347] Generated new cluster ID: bf91f622dc8647d3b5ec1a1fd0b57edc
I20260806 22:47:10.861953 13696 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:47:10.866030 13709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:10.861444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39226 (local address 127.6.40.126:37373)
0806 22:47:10.861698 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:10.861707 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:10.861761 (+ 54us) server_negotiation.cc:407] Connection header received
0806 22:47:10.861871 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:10.861878 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:10.861946 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:10.862077 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:10.862319 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:10.863191 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:10.864006 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:10.864456 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:10.864587 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:10.865301 (+ 714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:10.865305 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:10.865305 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:47:10.865492 (+ 187us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:10.865524 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:47:10.865535 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:10.865606 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:10.865680 (+ 74us) server_negotiation.cc:299] Negotiation successful
0806 22:47:10.865850 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":144,"threads_started":1}
I20260806 22:47:10.866961 6305 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:47:10.871591 13696 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:47:10.872253 13696 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:47:10.886225 13696 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1715ac52324c460babc67288966e9a05: Generated new TSK 0
I20260806 22:47:10.886418 13696 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260806 22:47:10.914606 13717 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260806 22:47:10.914768 13717 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260806 22:47:10.914999 13709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:10.899089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39230 (local address 127.6.40.126:37373)
0806 22:47:10.899158 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:10.899160 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:10.899292 (+ 132us) server_negotiation.cc:407] Connection header received
0806 22:47:10.899378 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:10.899379 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:10.899392 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:10.899477 (+ 85us) server_negotiation.cc:226] Negotiated authn=JWT
0806 22:47:10.899690 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:10.900477 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:10.901324 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:10.901613 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:10.901689 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:10.901877 (+ 188us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 22:47:10.914760 (+ 12883us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45821/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0806 22:47:10.914924 (+ 164us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:39230: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45821/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":41}
W20260806 22:47:10.915077 13709 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:39230: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45821/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260806 22:47:10.915539 13716 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:47:10.898988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:37373 (local address 127.0.0.1:39230)
0806 22:47:10.899221 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:10.899231 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:10.899293 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:10.899533 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:10.899535 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:10.899537 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0806 22:47:10.899605 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:10.899607 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:10.900574 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:10.900575 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:10.901243 (+ 668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:10.901246 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:10.901697 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:10.901698 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:10.901745 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:10.901765 (+ 20us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 22:47:10.914971 (+ 13206us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45821/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0806 22:47:10.915103 (+ 132us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.6.40.126:37373: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45821/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":126,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.6.40.126:37373: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45821/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260806 22:47:10.916335 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 13581
2026-08-06T22:47:10Z chronyd exiting
I20260806 22:47:10.959308 6305 test_util.cc:175] -----------------------------------------------
I20260806 22:47:10.959426 6305 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786056405724609-6305-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (235 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260806 22:47:10.960240 6305 mini_oidc.cc:152] Starting JWKS server
I20260806 22:47:10.961055 6305 webserver.cc:460] Webserver started at https://127.0.0.1:37869/ using document root <none> and password file <none>
I20260806 22:47:10.961123 6305 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 22:47:10.961295 6305 webserver.cc:460] Webserver started at http://127.0.0.1:46843/ using document root <none> and password file <none>
2026-08-06T22:47:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:47:10Z Disabled control of system clock
I20260806 22:47:10.970256 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786056405724609-6305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786056405724609-6305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786056405724609-6305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.126:35699
--webserver_interface=127.6.40.126
--webserver_port=0
--builtin_ntp_servers=127.6.40.84:41169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.40.126:35699
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37869/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786056405724609-6305-0/testchainca.cert with env {}
W20260806 22:47:11.052639 13726 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:47:11.052860 13726 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:47:11.052882 13726 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:47:11.054013 13726 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:47:11.054054 13726 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:47:11.054066 13726 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:47:11.054080 13726 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:47:11.054091 13726 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 22:47:11.054103 13726 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 22:47:11.054154 13726 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:37869/jwks.json
I20260806 22:47:11.055689 13726 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:41169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786056405724609-6305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786056405724609-6305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.40.126:35699
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.126:35699
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37869/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.40.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786056405724609-6305-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13726
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786056405724609-6305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:47:11.055887 13726 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:47:11.056043 13726 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 22:47:11.061050 13726 server_base.cc:1034] running on GCE node
W20260806 22:47:11.061002 13737 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:11.061023 13733 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:11.061010 13739 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:11.061457 13726 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:47:11.061762 13726 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:47:11.062904 13726 hybrid_clock.cc:648] HybridClock initialized: now 1786056431062883 us; error 34 us; skew 500 ppm
I20260806 22:47:11.064162 13726 webserver.cc:460] Webserver started at http://127.6.40.126:40355/ using document root <none> and password file <none>
I20260806 22:47:11.064433 13726 fs_manager.cc:359] Metadata directory not provided
I20260806 22:47:11.064484 13726 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:47:11.064590 13726 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:47:11.065660 13726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786056405724609-6305-0/minicluster-data/master-0/data/instance:
uuid: "2a85563b63d44221ac2e7cadb6be0609"
format_stamp: "Formatted at 2026-08-06 22:47:11 on dist-test-slave-zh2d"
server_key: "51ed925e2ea387917a8e0eabc88ce692"
server_key_iv: "e3f2af9c255f28dacfa5b0084481e455"
server_key_version: "encryptionkey@0"
I20260806 22:47:11.065956 13726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786056405724609-6305-0/minicluster-data/master-0/wal/instance:
uuid: "2a85563b63d44221ac2e7cadb6be0609"
format_stamp: "Formatted at 2026-08-06 22:47:11 on dist-test-slave-zh2d"
server_key: "51ed925e2ea387917a8e0eabc88ce692"
server_key_iv: "e3f2af9c255f28dacfa5b0084481e455"
server_key_version: "encryptionkey@0"
I20260806 22:47:11.067219 13726 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 22:47:11.067900 13748 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:11.068102 13726 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:47:11.068189 13726 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786056405724609-6305-0/minicluster-data/master-0/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786056405724609-6305-0/minicluster-data/master-0/wal
uuid: "2a85563b63d44221ac2e7cadb6be0609"
format_stamp: "Formatted at 2026-08-06 22:47:11 on dist-test-slave-zh2d"
server_key: "51ed925e2ea387917a8e0eabc88ce692"
server_key_iv: "e3f2af9c255f28dacfa5b0084481e455"
server_key_version: "encryptionkey@0"
I20260806 22:47:11.068245 13726 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786056405724609-6305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786056405724609-6305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786056405724609-6305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:47:11.095309 13726 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:47:11.095624 13726 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:47:11.095733 13726 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:47:11.099848 13726 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.126:35699
I20260806 22:47:11.099907 13824 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.126:35699 every 8 connection(s)
I20260806 22:47:11.100266 13726 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
I20260806 22:47:11.101423 13825 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:47:11.103842 13825 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a85563b63d44221ac2e7cadb6be0609: Bootstrap starting.
I20260806 22:47:11.104446 13825 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2a85563b63d44221ac2e7cadb6be0609: Neither blocks nor log segments found. Creating new log.
I20260806 22:47:11.104758 13825 log.cc:824] T 00000000000000000000000000000000 P 2a85563b63d44221ac2e7cadb6be0609: Log is configured to *not* fsync() on all Append() calls
I20260806 22:47:11.105053 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 13726
I20260806 22:47:11.105154 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786056405724609-6305-0/minicluster-data/master-0/wal/instance
I20260806 22:47:11.105376 6305 external_mini_cluster.cc:1428] Setting key 7bc7b8740489adbb50a42481e2a6ccb8
I20260806 22:47:11.105659 13825 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a85563b63d44221ac2e7cadb6be0609: No bootstrap required, opened a new log
I20260806 22:47:11.107435 13825 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2a85563b63d44221ac2e7cadb6be0609 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a85563b63d44221ac2e7cadb6be0609" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 35699 } }
I20260806 22:47:11.107573 13825 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2a85563b63d44221ac2e7cadb6be0609 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:47:11.107596 13825 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2a85563b63d44221ac2e7cadb6be0609 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a85563b63d44221ac2e7cadb6be0609, State: Initialized, Role: FOLLOWER
I20260806 22:47:11.107764 13825 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2a85563b63d44221ac2e7cadb6be0609 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a85563b63d44221ac2e7cadb6be0609" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 35699 } }
I20260806 22:47:11.107851 13825 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2a85563b63d44221ac2e7cadb6be0609 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:47:11.107890 13825 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2a85563b63d44221ac2e7cadb6be0609 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:47:11.107959 13825 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2a85563b63d44221ac2e7cadb6be0609 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:47:11.108500 13825 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2a85563b63d44221ac2e7cadb6be0609 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a85563b63d44221ac2e7cadb6be0609" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 35699 } }
I20260806 22:47:11.108589 13825 leader_election.cc:302] T 00000000000000000000000000000000 P 2a85563b63d44221ac2e7cadb6be0609 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2a85563b63d44221ac2e7cadb6be0609; no voters:
I20260806 22:47:11.108811 13825 leader_election.cc:288] T 00000000000000000000000000000000 P 2a85563b63d44221ac2e7cadb6be0609 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:47:11.108855 13831 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2a85563b63d44221ac2e7cadb6be0609 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:47:11.108986 13831 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2a85563b63d44221ac2e7cadb6be0609 [term 1 LEADER]: Becoming Leader. State: Replica: 2a85563b63d44221ac2e7cadb6be0609, State: Running, Role: LEADER
I20260806 22:47:11.109066 13831 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2a85563b63d44221ac2e7cadb6be0609 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a85563b63d44221ac2e7cadb6be0609" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 35699 } }
I20260806 22:47:11.109109 13825 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2a85563b63d44221ac2e7cadb6be0609 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:47:11.109422 13831 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a85563b63d44221ac2e7cadb6be0609 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2a85563b63d44221ac2e7cadb6be0609" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a85563b63d44221ac2e7cadb6be0609" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 35699 } } }
I20260806 22:47:11.109503 13831 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a85563b63d44221ac2e7cadb6be0609 [sys.catalog]: This master's current role is: LEADER
I20260806 22:47:11.109664 13831 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a85563b63d44221ac2e7cadb6be0609 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2a85563b63d44221ac2e7cadb6be0609. Latest consensus state: current_term: 1 leader_uuid: "2a85563b63d44221ac2e7cadb6be0609" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a85563b63d44221ac2e7cadb6be0609" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 35699 } } }
I20260806 22:47:11.109738 13831 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a85563b63d44221ac2e7cadb6be0609 [sys.catalog]: This master's current role is: LEADER
W20260806 22:47:11.110937 13852 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2a85563b63d44221ac2e7cadb6be0609: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:47:11.110994 13852 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:47:11.111047 13836 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:47:11.111235 13836 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:47:11.112964 13836 catalog_manager.cc:1347] Generated new cluster ID: 664117b84e2c4fed998e6a3fafd0ae0b
I20260806 22:47:11.113031 13836 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:47:11.113133 13830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:11.105948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38564 (local address 127.6.40.126:35699)
0806 22:47:11.106195 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:11.106205 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:11.106240 (+ 35us) server_negotiation.cc:407] Connection header received
0806 22:47:11.107419 (+ 1179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:11.107426 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:11.107506 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:11.107661 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:11.107971 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:11.109608 (+ 1637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:11.110784 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:11.111387 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:11.111483 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:11.112279 (+ 796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:11.112283 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:11.112283 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:47:11.112522 (+ 239us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:11.112567 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:47:11.112586 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:11.112661 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:11.112739 (+ 78us) server_negotiation.cc:299] Negotiation successful
0806 22:47:11.112820 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":97,"threads_started":1}
I20260806 22:47:11.114028 6305 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:47:11.119529 13836 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:47:11.120043 13836 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:47:11.123080 13836 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2a85563b63d44221ac2e7cadb6be0609: Generated new TSK 0
I20260806 22:47:11.123248 13836 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260806 22:47:11.144857 13863 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260806 22:47:11.144945 13863 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260806 22:47:11.145252 13830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:11.130488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38572 (local address 127.6.40.126:35699)
0806 22:47:11.130548 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:11.130551 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:11.130660 (+ 109us) server_negotiation.cc:407] Connection header received
0806 22:47:11.130729 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:11.130730 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:11.130740 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:11.130795 (+ 55us) server_negotiation.cc:226] Negotiated authn=JWT
0806 22:47:11.130913 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:11.131395 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:11.132552 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:11.132734 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:11.132783 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:11.133003 (+ 220us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 22:47:11.145088 (+ 12085us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37869/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0806 22:47:11.145205 (+ 117us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:38572: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37869/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":36}
W20260806 22:47:11.145323 13830 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:38572: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37869/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260806 22:47:11.145505 13862 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:47:11.130421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:35699 (local address 127.0.0.1:38572)
0806 22:47:11.130584 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:11.130595 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:11.130662 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:11.130796 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:11.130797 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:11.130798 (+ 1us) client_negotiation.cc:189] Negotiated authn=JWT
0806 22:47:11.130855 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:11.130857 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:11.131491 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:11.131492 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:11.132467 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:11.132469 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:11.132808 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:11.132809 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:11.132900 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:11.132906 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 22:47:11.145198 (+ 12292us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37869/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0806 22:47:11.145320 (+ 122us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.6.40.126:35699: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37869/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":108,"threads_started":1}
I20260806 22:47:11.145839 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 13726
2026-08-06T22:47:11Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (232 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260806 22:47:11.192765 6305 mini_oidc.cc:152] Starting JWKS server
I20260806 22:47:11.193532 6305 webserver.cc:460] Webserver started at https://127.0.0.1:43897/ using document root <none> and password file <none>
I20260806 22:47:11.193580 6305 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 22:47:11.193722 6305 webserver.cc:460] Webserver started at http://127.0.0.1:43833/ using document root <none> and password file <none>
2026-08-06T22:47:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:47:11Z Disabled control of system clock
I20260806 22:47:11.202375 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786056405724609-6305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786056405724609-6305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786056405724609-6305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.126:46607
--webserver_interface=127.6.40.126
--webserver_port=0
--builtin_ntp_servers=127.6.40.84:42099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.40.126:46607
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43897/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260806 22:47:11.290169 13870 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:47:11.290436 13870 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:47:11.290478 13870 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:47:11.291692 13870 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:47:11.291764 13870 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:47:11.291788 13870 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:47:11.291814 13870 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:47:11.291849 13870 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 22:47:11.291879 13870 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 22:47:11.291910 13870 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:43897/jwks.json
I20260806 22:47:11.293370 13870 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:42099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786056405724609-6305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786056405724609-6305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.40.126:46607
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.126:46607
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43897/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13870
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786056405724609-6305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:47:11.293661 13870 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:47:11.293993 13870 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:47:11.299330 13882 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:11.299335 13879 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:11.299505 13870 server_base.cc:1034] running on GCE node
W20260806 22:47:11.299424 13878 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:11.299849 13870 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:47:11.300168 13870 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:47:11.301321 13870 hybrid_clock.cc:648] HybridClock initialized: now 1786056431301301 us; error 41 us; skew 500 ppm
I20260806 22:47:11.302675 13870 webserver.cc:460] Webserver started at http://127.6.40.126:37971/ using document root <none> and password file <none>
I20260806 22:47:11.302920 13870 fs_manager.cc:359] Metadata directory not provided
I20260806 22:47:11.302974 13870 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:47:11.303154 13870 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:47:11.304363 13870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786056405724609-6305-0/minicluster-data/master-0/data/instance:
uuid: "270f874350a24008a766adb4bebd80f6"
format_stamp: "Formatted at 2026-08-06 22:47:11 on dist-test-slave-zh2d"
server_key: "4496ee9c30b44c4a01ad6edc785bfc1c"
server_key_iv: "8b7d7440e72402027c4f65d4cb439f0b"
server_key_version: "encryptionkey@0"
I20260806 22:47:11.304764 13870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786056405724609-6305-0/minicluster-data/master-0/wal/instance:
uuid: "270f874350a24008a766adb4bebd80f6"
format_stamp: "Formatted at 2026-08-06 22:47:11 on dist-test-slave-zh2d"
server_key: "4496ee9c30b44c4a01ad6edc785bfc1c"
server_key_iv: "8b7d7440e72402027c4f65d4cb439f0b"
server_key_version: "encryptionkey@0"
I20260806 22:47:11.306092 13870 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:47:11.306846 13890 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:11.307044 13870 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:47:11.307152 13870 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786056405724609-6305-0/minicluster-data/master-0/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786056405724609-6305-0/minicluster-data/master-0/wal
uuid: "270f874350a24008a766adb4bebd80f6"
format_stamp: "Formatted at 2026-08-06 22:47:11 on dist-test-slave-zh2d"
server_key: "4496ee9c30b44c4a01ad6edc785bfc1c"
server_key_iv: "8b7d7440e72402027c4f65d4cb439f0b"
server_key_version: "encryptionkey@0"
I20260806 22:47:11.307228 13870 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786056405724609-6305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786056405724609-6305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786056405724609-6305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:47:11.315984 13870 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:47:11.316286 13870 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:47:11.316435 13870 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:47:11.320605 13870 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.126:46607
I20260806 22:47:11.320640 13958 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.126:46607 every 8 connection(s)
I20260806 22:47:11.321033 13870 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
I20260806 22:47:11.321702 13960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:47:11.324611 13960 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 270f874350a24008a766adb4bebd80f6: Bootstrap starting.
I20260806 22:47:11.325282 13960 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 270f874350a24008a766adb4bebd80f6: Neither blocks nor log segments found. Creating new log.
I20260806 22:47:11.325601 13960 log.cc:824] T 00000000000000000000000000000000 P 270f874350a24008a766adb4bebd80f6: Log is configured to *not* fsync() on all Append() calls
I20260806 22:47:11.326292 13960 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 270f874350a24008a766adb4bebd80f6: No bootstrap required, opened a new log
I20260806 22:47:11.327209 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 13870
I20260806 22:47:11.327301 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786056405724609-6305-0/minicluster-data/master-0/wal/instance
I20260806 22:47:11.327512 6305 external_mini_cluster.cc:1428] Setting key 6ebcc4b61a9e66602b8744f65271d636
I20260806 22:47:11.328395 13960 raft_consensus.cc:348] T 00000000000000000000000000000000 P 270f874350a24008a766adb4bebd80f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "270f874350a24008a766adb4bebd80f6" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 46607 } }
I20260806 22:47:11.328573 13960 raft_consensus.cc:374] T 00000000000000000000000000000000 P 270f874350a24008a766adb4bebd80f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:47:11.328617 13960 raft_consensus.cc:729] T 00000000000000000000000000000000 P 270f874350a24008a766adb4bebd80f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 270f874350a24008a766adb4bebd80f6, State: Initialized, Role: FOLLOWER
I20260806 22:47:11.328780 13960 consensus_queue.cc:260] T 00000000000000000000000000000000 P 270f874350a24008a766adb4bebd80f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "270f874350a24008a766adb4bebd80f6" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 46607 } }
I20260806 22:47:11.328889 13960 raft_consensus.cc:388] T 00000000000000000000000000000000 P 270f874350a24008a766adb4bebd80f6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:47:11.328933 13960 raft_consensus.cc:482] T 00000000000000000000000000000000 P 270f874350a24008a766adb4bebd80f6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:47:11.328981 13960 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 270f874350a24008a766adb4bebd80f6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:47:11.329550 13960 raft_consensus.cc:504] T 00000000000000000000000000000000 P 270f874350a24008a766adb4bebd80f6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "270f874350a24008a766adb4bebd80f6" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 46607 } }
I20260806 22:47:11.329641 13960 leader_election.cc:302] T 00000000000000000000000000000000 P 270f874350a24008a766adb4bebd80f6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 270f874350a24008a766adb4bebd80f6; no voters:
I20260806 22:47:11.329836 13960 leader_election.cc:288] T 00000000000000000000000000000000 P 270f874350a24008a766adb4bebd80f6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:47:11.329873 13968 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 270f874350a24008a766adb4bebd80f6 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:47:11.330001 13968 raft_consensus.cc:686] T 00000000000000000000000000000000 P 270f874350a24008a766adb4bebd80f6 [term 1 LEADER]: Becoming Leader. State: Replica: 270f874350a24008a766adb4bebd80f6, State: Running, Role: LEADER
I20260806 22:47:11.330087 13968 consensus_queue.cc:237] T 00000000000000000000000000000000 P 270f874350a24008a766adb4bebd80f6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "270f874350a24008a766adb4bebd80f6" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 46607 } }
I20260806 22:47:11.330118 13960 sys_catalog.cc:564] T 00000000000000000000000000000000 P 270f874350a24008a766adb4bebd80f6 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:47:11.330442 13968 sys_catalog.cc:455] T 00000000000000000000000000000000 P 270f874350a24008a766adb4bebd80f6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 270f874350a24008a766adb4bebd80f6. Latest consensus state: current_term: 1 leader_uuid: "270f874350a24008a766adb4bebd80f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "270f874350a24008a766adb4bebd80f6" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 46607 } } }
I20260806 22:47:11.330466 13970 sys_catalog.cc:455] T 00000000000000000000000000000000 P 270f874350a24008a766adb4bebd80f6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "270f874350a24008a766adb4bebd80f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "270f874350a24008a766adb4bebd80f6" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 46607 } } }
I20260806 22:47:11.330524 13968 sys_catalog.cc:458] T 00000000000000000000000000000000 P 270f874350a24008a766adb4bebd80f6 [sys.catalog]: This master's current role is: LEADER
I20260806 22:47:11.330543 13970 sys_catalog.cc:458] T 00000000000000000000000000000000 P 270f874350a24008a766adb4bebd80f6 [sys.catalog]: This master's current role is: LEADER
W20260806 22:47:11.332019 13991 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 270f874350a24008a766adb4bebd80f6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:47:11.332077 13991 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:47:11.332131 13975 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:47:11.332275 13975 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:47:11.334097 13967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:11.328037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59972 (local address 127.6.40.126:46607)
0806 22:47:11.328280 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:11.328289 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:11.328326 (+ 37us) server_negotiation.cc:407] Connection header received
0806 22:47:11.328444 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:11.328451 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:11.328546 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:11.328694 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:11.329010 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:11.330672 (+ 1662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:11.331791 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:11.332411 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:11.332533 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:11.333293 (+ 760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:11.333297 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:11.333297 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:47:11.333562 (+ 265us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:11.333605 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:47:11.333626 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:11.333704 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:11.333783 (+ 79us) server_negotiation.cc:299] Negotiation successful
0806 22:47:11.333867 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":98,"threads_started":1}
I20260806 22:47:11.334236 13975 catalog_manager.cc:1347] Generated new cluster ID: 2eb38027dcb44f4dbdfb255d2fb5a211
I20260806 22:47:11.334280 13975 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:47:11.335041 6305 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:47:11.345621 13975 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:47:11.346154 13975 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:47:11.352025 13975 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 270f874350a24008a766adb4bebd80f6: Generated new TSK 0
I20260806 22:47:11.352216 13975 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:47:11.363335 13967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:11.359584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59988 (local address 127.6.40.126:46607)
0806 22:47:11.359633 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:11.359635 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:11.359741 (+ 106us) server_negotiation.cc:407] Connection header received
0806 22:47:11.359810 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:11.359810 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:11.359821 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:11.359882 (+ 61us) server_negotiation.cc:226] Negotiated authn=JWT
0806 22:47:11.360046 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:11.360554 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:11.361734 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:11.361917 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:11.361973 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:11.362168 (+ 195us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 22:47:11.363120 (+ 952us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43897/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0806 22:47:11.363256 (+ 136us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:59988: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43897/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":26}
W20260806 22:47:11.363400 13967 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:59988: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43897/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260806 22:47:11.363735 14000 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:47:11.359484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:46607 (local address 127.0.0.1:59988)
0806 22:47:11.359662 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:11.359674 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:11.359742 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:11.359906 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:11.359907 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:11.359909 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0806 22:47:11.359976 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:11.359980 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:11.360640 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:11.360641 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:11.361646 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:11.361649 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:11.362004 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:11.362005 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:11.362069 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:11.362075 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 22:47:11.363233 (+ 1158us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43897/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0806 22:47:11.363344 (+ 111us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.6.40.126:46607: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43897/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":105,"threads_started":1}
I20260806 22:47:11.365433 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 13870
E20260806 22:47:11.377333 14001 webserver.cc:533] Webserver: sslize failed:
E20260806 22:47:11.377414 14001 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-06T22:47:11Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (214 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-06T22:47:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:47:11Z Disabled control of system clock
I20260806 22:47:11.415853 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786056405724609-6305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786056405724609-6305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786056405724609-6305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.126:41385
--webserver_interface=127.6.40.126
--webserver_port=0
--builtin_ntp_servers=127.6.40.84:44577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.40.126:41385
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786056405724609-6305-0/testchainca.cert with env {}
W20260806 22:47:11.497915 14006 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:47:11.498122 14006 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:47:11.498147 14006 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:47:11.499344 14006 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:47:11.499418 14006 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:47:11.499459 14006 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:47:11.499500 14006 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:47:11.499528 14006 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 22:47:11.499565 14006 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 22:47:11.499596 14006 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260806 22:47:11.501488 14006 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:44577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786056405724609-6305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786056405724609-6305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.40.126:41385
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.126:41385
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.40.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786056405724609-6305-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14006
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786056405724609-6305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:47:11.501777 14006 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:47:11.502034 14006 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:47:11.507203 14016 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:11.507216 14015 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:11.507409 14006 server_base.cc:1034] running on GCE node
W20260806 22:47:11.507216 14018 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:11.507803 14006 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:47:11.508123 14006 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:47:11.509292 14006 hybrid_clock.cc:648] HybridClock initialized: now 1786056431509265 us; error 46 us; skew 500 ppm
I20260806 22:47:11.510692 14006 webserver.cc:460] Webserver started at http://127.6.40.126:40841/ using document root <none> and password file <none>
I20260806 22:47:11.510954 14006 fs_manager.cc:359] Metadata directory not provided
I20260806 22:47:11.511029 14006 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:47:11.511189 14006 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:47:11.512288 14006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786056405724609-6305-0/minicluster-data/master-0/data/instance:
uuid: "57b5b085402c4c1d82da6332ebf28a2d"
format_stamp: "Formatted at 2026-08-06 22:47:11 on dist-test-slave-zh2d"
server_key: "2822f8dd6bd34388046ff26dff39c0cb"
server_key_iv: "379a5880aadd67c97e5d07a4b696717d"
server_key_version: "encryptionkey@0"
I20260806 22:47:11.512671 14006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786056405724609-6305-0/minicluster-data/master-0/wal/instance:
uuid: "57b5b085402c4c1d82da6332ebf28a2d"
format_stamp: "Formatted at 2026-08-06 22:47:11 on dist-test-slave-zh2d"
server_key: "2822f8dd6bd34388046ff26dff39c0cb"
server_key_iv: "379a5880aadd67c97e5d07a4b696717d"
server_key_version: "encryptionkey@0"
I20260806 22:47:11.514076 14006 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:47:11.514850 14024 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:11.515031 14006 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:47:11.515142 14006 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786056405724609-6305-0/minicluster-data/master-0/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786056405724609-6305-0/minicluster-data/master-0/wal
uuid: "57b5b085402c4c1d82da6332ebf28a2d"
format_stamp: "Formatted at 2026-08-06 22:47:11 on dist-test-slave-zh2d"
server_key: "2822f8dd6bd34388046ff26dff39c0cb"
server_key_iv: "379a5880aadd67c97e5d07a4b696717d"
server_key_version: "encryptionkey@0"
I20260806 22:47:11.515226 14006 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786056405724609-6305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786056405724609-6305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786056405724609-6305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:47:11.526484 14006 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:47:11.526808 14006 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:47:11.526966 14006 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:47:11.531229 14006 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.126:41385
I20260806 22:47:11.531262 14095 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.126:41385 every 8 connection(s)
I20260806 22:47:11.531618 14006 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
I20260806 22:47:11.532615 14096 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:47:11.535033 14096 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57b5b085402c4c1d82da6332ebf28a2d: Bootstrap starting.
I20260806 22:47:11.535629 14096 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 57b5b085402c4c1d82da6332ebf28a2d: Neither blocks nor log segments found. Creating new log.
I20260806 22:47:11.535931 14096 log.cc:824] T 00000000000000000000000000000000 P 57b5b085402c4c1d82da6332ebf28a2d: Log is configured to *not* fsync() on all Append() calls
I20260806 22:47:11.536682 14096 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57b5b085402c4c1d82da6332ebf28a2d: No bootstrap required, opened a new log
I20260806 22:47:11.538373 14096 raft_consensus.cc:348] T 00000000000000000000000000000000 P 57b5b085402c4c1d82da6332ebf28a2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57b5b085402c4c1d82da6332ebf28a2d" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 41385 } }
I20260806 22:47:11.538511 14096 raft_consensus.cc:374] T 00000000000000000000000000000000 P 57b5b085402c4c1d82da6332ebf28a2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:47:11.538537 14096 raft_consensus.cc:729] T 00000000000000000000000000000000 P 57b5b085402c4c1d82da6332ebf28a2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57b5b085402c4c1d82da6332ebf28a2d, State: Initialized, Role: FOLLOWER
I20260806 22:47:11.538694 14096 consensus_queue.cc:260] T 00000000000000000000000000000000 P 57b5b085402c4c1d82da6332ebf28a2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57b5b085402c4c1d82da6332ebf28a2d" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 41385 } }
I20260806 22:47:11.538757 14096 raft_consensus.cc:388] T 00000000000000000000000000000000 P 57b5b085402c4c1d82da6332ebf28a2d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:47:11.538779 14096 raft_consensus.cc:482] T 00000000000000000000000000000000 P 57b5b085402c4c1d82da6332ebf28a2d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:47:11.538841 14096 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 57b5b085402c4c1d82da6332ebf28a2d [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:47:11.539397 14096 raft_consensus.cc:504] T 00000000000000000000000000000000 P 57b5b085402c4c1d82da6332ebf28a2d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57b5b085402c4c1d82da6332ebf28a2d" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 41385 } }
I20260806 22:47:11.539482 14096 leader_election.cc:302] T 00000000000000000000000000000000 P 57b5b085402c4c1d82da6332ebf28a2d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 57b5b085402c4c1d82da6332ebf28a2d; no voters:
I20260806 22:47:11.539693 14096 leader_election.cc:288] T 00000000000000000000000000000000 P 57b5b085402c4c1d82da6332ebf28a2d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:47:11.539736 14100 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 57b5b085402c4c1d82da6332ebf28a2d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:47:11.539927 14100 raft_consensus.cc:686] T 00000000000000000000000000000000 P 57b5b085402c4c1d82da6332ebf28a2d [term 1 LEADER]: Becoming Leader. State: Replica: 57b5b085402c4c1d82da6332ebf28a2d, State: Running, Role: LEADER
I20260806 22:47:11.539956 14096 sys_catalog.cc:564] T 00000000000000000000000000000000 P 57b5b085402c4c1d82da6332ebf28a2d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:47:11.540025 14100 consensus_queue.cc:237] T 00000000000000000000000000000000 P 57b5b085402c4c1d82da6332ebf28a2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57b5b085402c4c1d82da6332ebf28a2d" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 41385 } }
I20260806 22:47:11.540467 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 14006
I20260806 22:47:11.540602 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786056405724609-6305-0/minicluster-data/master-0/wal/instance
I20260806 22:47:11.540721 14105 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57b5b085402c4c1d82da6332ebf28a2d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 57b5b085402c4c1d82da6332ebf28a2d. Latest consensus state: current_term: 1 leader_uuid: "57b5b085402c4c1d82da6332ebf28a2d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57b5b085402c4c1d82da6332ebf28a2d" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 41385 } } }
I20260806 22:47:11.540808 14105 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57b5b085402c4c1d82da6332ebf28a2d [sys.catalog]: This master's current role is: LEADER
I20260806 22:47:11.540928 6305 external_mini_cluster.cc:1428] Setting key 0208d2f741f969a22e45d847d513eae1
I20260806 22:47:11.541004 14101 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57b5b085402c4c1d82da6332ebf28a2d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "57b5b085402c4c1d82da6332ebf28a2d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57b5b085402c4c1d82da6332ebf28a2d" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 41385 } } }
I20260806 22:47:11.541071 14101 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57b5b085402c4c1d82da6332ebf28a2d [sys.catalog]: This master's current role is: LEADER
I20260806 22:47:11.541211 14112 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:47:11.541736 14112 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:47:11.543426 14112 catalog_manager.cc:1347] Generated new cluster ID: de77252fa3b04f3cb2a7291babf991be
I20260806 22:47:11.543489 14112 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:47:11.547205 14123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:11.541512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38746 (local address 127.6.40.126:41385)
0806 22:47:11.541807 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:11.541817 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:11.541853 (+ 36us) server_negotiation.cc:407] Connection header received
0806 22:47:11.541980 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:11.541988 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:11.542068 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:11.542282 (+ 214us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:11.542509 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:11.543850 (+ 1341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:11.544968 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:11.545617 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:11.545754 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:11.546434 (+ 680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:11.546438 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:11.546438 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:47:11.546675 (+ 237us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:11.546717 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:47:11.546731 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:11.546819 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:11.546909 (+ 90us) server_negotiation.cc:299] Negotiation successful
0806 22:47:11.546990 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":212,"thread_start_us":71,"threads_started":1}
I20260806 22:47:11.548127 6305 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:47:11.553745 14112 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:47:11.554342 14112 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:47:11.558574 14112 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 57b5b085402c4c1d82da6332ebf28a2d: Generated new TSK 0
I20260806 22:47:11.558754 14112 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:47:12.046175 14123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:11.567233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38752 (local address 127.6.40.126:41385)
0806 22:47:11.567305 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:11.567307 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:11.567437 (+ 130us) server_negotiation.cc:407] Connection header received
0806 22:47:11.567485 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:11.567486 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:11.567512 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:11.567594 (+ 82us) server_negotiation.cc:226] Negotiated authn=JWT
0806 22:47:11.567723 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:11.568548 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:11.569387 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:11.569727 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:11.569799 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:11.569931 (+ 132us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 22:47:12.045937 (+476006us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0806 22:47:12.046108 (+ 171us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:38752: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":46}
W20260806 22:47:12.046242 14123 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:38752: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260806 22:47:12.046365 14134 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:47:11.567156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:41385 (local address 127.0.0.1:38752)
0806 22:47:11.567364 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:11.567373 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:11.567430 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:11.567601 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:11.567602 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:11.567603 (+ 1us) client_negotiation.cc:189] Negotiated authn=JWT
0806 22:47:11.567658 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:11.567660 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:11.568649 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:11.568650 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:11.569306 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:11.569309 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:11.569810 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:11.569810 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:11.569851 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:11.569856 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 22:47:12.046086 (+476230us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0806 22:47:12.046221 (+ 135us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.6.40.126:41385: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":114,"threads_started":1}
I20260806 22:47:12.046774 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 14006
2026-08-06T22:47:12Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (665 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (92 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (94 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:47:12 dist-test-slave-zh2d krb5kdc[14148](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:47:12 dist-test-slave-zh2d krb5kdc[14148](info): set up 2 sockets
Aug 06 22:47:12 dist-test-slave-zh2d krb5kdc[14148](info): commencing operation
krb5kdc: starting...
W20260806 22:47:13.543411 6305 mini_kdc.cc:121] Time spent starting KDC: real 1.284s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
W20260806 22:47:13.756681 6305 mini_kdc.cc:253] Time spent creating user principal test-admin: real 0.213s user 0.000s sys 0.003s
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
W20260806 22:47:13.895282 6305 mini_kdc.cc:253] Time spent creating user principal test-user: real 0.138s user 0.002s sys 0.000s
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
W20260806 22:47:14.029986 6305 mini_kdc.cc:253] Time spent creating user principal joe-interloper: real 0.135s user 0.001s sys 0.002s
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/krb5kdc/test-user.keytab.
W20260806 22:47:14.673156 6305 mini_kdc.cc:302] Time spent creating keytab for test-user: real 0.643s user 0.001s sys 0.002s
Aug 06 22:47:14 dist-test-slave-zh2d krb5kdc[14148](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056434, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:47:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:47:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.40.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.40.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.40.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab.
Entry for principal kudu/127.6.40.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab.
W20260806 22:47:14.904436 6305 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.6.40.126: real 0.205s user 0.000s sys 0.004s
I20260806 22:47:14.904889 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.126:34529
--webserver_interface=127.6.40.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab
--principal=kudu/127.6.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:34913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.40.126:34529
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:47:14.992796 14165 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:47:14.993100 14165 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:47:14.993148 14165 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:47:14.994401 14165 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:47:14.994500 14165 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:47:14.994526 14165 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:47:14.994549 14165 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:47:14.994578 14165 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:47:14.996145 14165 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:34913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.40.126:34529
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab
--principal=kudu/127.6.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.126:34529
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14165
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:47:14.996445 14165 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:47:14.996691 14165 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:47:15.002185 14174 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:15.002315 14178 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:15.002182 14181 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:15.002470 14165 server_base.cc:1034] running on GCE node
I20260806 22:47:15.002815 14165 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:47:15.003151 14165 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:47:15.004345 14165 hybrid_clock.cc:648] HybridClock initialized: now 1786056435004334 us; error 51 us; skew 500 ppm
Aug 06 22:47:15 dist-test-slave-zh2d krb5kdc[14148](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056435, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:47:15.010360 14165 init.cc:377] Logged in from keytab as kudu/127.6.40.126@KRBTEST.COM (short username kudu)
I20260806 22:47:15.010851 14165 webserver.cc:460] Webserver started at http://127.6.40.126:45375/ using document root <none> and password file <none>
I20260806 22:47:15.011099 14165 fs_manager.cc:359] Metadata directory not provided
I20260806 22:47:15.011168 14165 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:47:15.011317 14165 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:47:15.012465 14165 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/minicluster-data/master-0/data/instance:
uuid: "235898b1fa014563a9878fd7f1ea5cd5"
format_stamp: "Formatted at 2026-08-06 22:47:15 on dist-test-slave-zh2d"
server_key: "fda826d14f0de190ce2cc7959594932e"
server_key_iv: "57f034c18cd6bbbea77c971f124e91b9"
server_key_version: "encryptionkey@0"
I20260806 22:47:15.012825 14165 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/minicluster-data/master-0/wal/instance:
uuid: "235898b1fa014563a9878fd7f1ea5cd5"
format_stamp: "Formatted at 2026-08-06 22:47:15 on dist-test-slave-zh2d"
server_key: "fda826d14f0de190ce2cc7959594932e"
server_key_iv: "57f034c18cd6bbbea77c971f124e91b9"
server_key_version: "encryptionkey@0"
I20260806 22:47:15.014271 14165 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:47:15.015076 14194 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:15.015271 14165 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:47:15.015372 14165 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/minicluster-data/master-0/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/minicluster-data/master-0/wal
uuid: "235898b1fa014563a9878fd7f1ea5cd5"
format_stamp: "Formatted at 2026-08-06 22:47:15 on dist-test-slave-zh2d"
server_key: "fda826d14f0de190ce2cc7959594932e"
server_key_iv: "57f034c18cd6bbbea77c971f124e91b9"
server_key_version: "encryptionkey@0"
I20260806 22:47:15.015450 14165 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:47:15.027581 14165 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:47:15.030090 14165 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:47:15.030256 14165 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:47:15.034811 14165 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.126:34529
I20260806 22:47:15.034868 14270 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.126:34529 every 8 connection(s)
I20260806 22:47:15.035220 14165 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
I20260806 22:47:15.036123 14271 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:47:15.038522 14271 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 235898b1fa014563a9878fd7f1ea5cd5: Bootstrap starting.
I20260806 22:47:15.039069 14271 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 235898b1fa014563a9878fd7f1ea5cd5: Neither blocks nor log segments found. Creating new log.
I20260806 22:47:15.039315 14271 log.cc:824] T 00000000000000000000000000000000 P 235898b1fa014563a9878fd7f1ea5cd5: Log is configured to *not* fsync() on all Append() calls
I20260806 22:47:15.040059 14271 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 235898b1fa014563a9878fd7f1ea5cd5: No bootstrap required, opened a new log
I20260806 22:47:15.040748 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 14165
I20260806 22:47:15.040885 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/minicluster-data/master-0/wal/instance
I20260806 22:47:15.041097 6305 external_mini_cluster.cc:1428] Setting key d7820cfb6527cbbae406edbfbfbeb904
I20260806 22:47:15.042833 14271 raft_consensus.cc:348] T 00000000000000000000000000000000 P 235898b1fa014563a9878fd7f1ea5cd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "235898b1fa014563a9878fd7f1ea5cd5" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 34529 } }
I20260806 22:47:15.043032 14271 raft_consensus.cc:374] T 00000000000000000000000000000000 P 235898b1fa014563a9878fd7f1ea5cd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:47:15.043076 14271 raft_consensus.cc:729] T 00000000000000000000000000000000 P 235898b1fa014563a9878fd7f1ea5cd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235898b1fa014563a9878fd7f1ea5cd5, State: Initialized, Role: FOLLOWER
I20260806 22:47:15.043208 14271 consensus_queue.cc:260] T 00000000000000000000000000000000 P 235898b1fa014563a9878fd7f1ea5cd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "235898b1fa014563a9878fd7f1ea5cd5" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 34529 } }
I20260806 22:47:15.043294 14271 raft_consensus.cc:388] T 00000000000000000000000000000000 P 235898b1fa014563a9878fd7f1ea5cd5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:47:15.043329 14271 raft_consensus.cc:482] T 00000000000000000000000000000000 P 235898b1fa014563a9878fd7f1ea5cd5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:47:15.043370 14271 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 235898b1fa014563a9878fd7f1ea5cd5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:47:15.044036 14271 raft_consensus.cc:504] T 00000000000000000000000000000000 P 235898b1fa014563a9878fd7f1ea5cd5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "235898b1fa014563a9878fd7f1ea5cd5" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 34529 } }
I20260806 22:47:15.044128 14271 leader_election.cc:302] T 00000000000000000000000000000000 P 235898b1fa014563a9878fd7f1ea5cd5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 235898b1fa014563a9878fd7f1ea5cd5; no voters:
I20260806 22:47:15.044355 14271 leader_election.cc:288] T 00000000000000000000000000000000 P 235898b1fa014563a9878fd7f1ea5cd5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:47:15.044402 14276 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 235898b1fa014563a9878fd7f1ea5cd5 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:47:15.044575 14276 raft_consensus.cc:686] T 00000000000000000000000000000000 P 235898b1fa014563a9878fd7f1ea5cd5 [term 1 LEADER]: Becoming Leader. State: Replica: 235898b1fa014563a9878fd7f1ea5cd5, State: Running, Role: LEADER
I20260806 22:47:15.044644 14271 sys_catalog.cc:564] T 00000000000000000000000000000000 P 235898b1fa014563a9878fd7f1ea5cd5 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:47:15.044715 14276 consensus_queue.cc:237] T 00000000000000000000000000000000 P 235898b1fa014563a9878fd7f1ea5cd5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "235898b1fa014563a9878fd7f1ea5cd5" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 34529 } }
I20260806 22:47:15.045114 14276 sys_catalog.cc:455] T 00000000000000000000000000000000 P 235898b1fa014563a9878fd7f1ea5cd5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 235898b1fa014563a9878fd7f1ea5cd5. Latest consensus state: current_term: 1 leader_uuid: "235898b1fa014563a9878fd7f1ea5cd5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "235898b1fa014563a9878fd7f1ea5cd5" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 34529 } } }
I20260806 22:47:15.045354 14276 sys_catalog.cc:458] T 00000000000000000000000000000000 P 235898b1fa014563a9878fd7f1ea5cd5 [sys.catalog]: This master's current role is: LEADER
I20260806 22:47:15.045915 14288 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:47:15.046192 14280 sys_catalog.cc:455] T 00000000000000000000000000000000 P 235898b1fa014563a9878fd7f1ea5cd5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "235898b1fa014563a9878fd7f1ea5cd5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "235898b1fa014563a9878fd7f1ea5cd5" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 34529 } } }
I20260806 22:47:15.046399 14280 sys_catalog.cc:458] T 00000000000000000000000000000000 P 235898b1fa014563a9878fd7f1ea5cd5 [sys.catalog]: This master's current role is: LEADER
I20260806 22:47:15.046442 14288 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:47:15.048401 14288 catalog_manager.cc:1347] Generated new cluster ID: f139bad61033470495b4a91f9803f607
I20260806 22:47:15.048458 14288 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 22:47:15 dist-test-slave-zh2d krb5kdc[14148](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056434, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
I20260806 22:47:15.053098 14275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:15.041638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37474 (local address 127.6.40.126:34529)
0806 22:47:15.041815 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:15.041823 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:15.041864 (+ 41us) server_negotiation.cc:407] Connection header received
0806 22:47:15.042761 (+ 897us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:15.042769 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:15.042817 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:15.042937 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:15.043800 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:15.045028 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:15.046140 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:15.046920 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:15.047033 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:15.049808 (+ 2775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:15.049812 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:15.049813 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:15.049825 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:15.051599 (+ 1774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:15.051942 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:15.051945 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:15.051946 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:15.052037 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:15.052256 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:15.052257 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:15.052257 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:15.052492 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:15.052677 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:15.052746 (+ 69us) server_negotiation.cc:299] Negotiation successful
0806 22:47:15.052908 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":92,"threads_started":1}
I20260806 22:47:15.056820 14288 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:47:15.057538 14288 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:47:15.069856 14288 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 235898b1fa014563a9878fd7f1ea5cd5: Generated new TSK 0
I20260806 22:47:15.070053 14288 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.40.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.40.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.40.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.65.keytab.
Entry for principal kudu/127.6.40.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.65.keytab.
I20260806 22:47:15.100540 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.65:0
--local_ip_for_outbound_sockets=127.6.40.65
--webserver_interface=127.6.40.65
--webserver_port=0
--tserver_master_addrs=127.6.40.126:34529
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.65.keytab
--principal=kudu/127.6.40.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:34913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:47:15.186659 14305 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:47:15.186908 14305 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:47:15.186950 14305 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:47:15.188217 14305 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:47:15.188289 14305 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:47:15.188334 14305 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.65
I20260806 22:47:15.189935 14305 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:34913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.65.keytab
--principal=kudu/127.6.40.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.40.65
--webserver_port=0
--tserver_master_addrs=127.6.40.126:34529
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14305
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.65
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:47:15.190210 14305 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:47:15.190447 14305 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:47:15.195766 14315 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:15.195762 14312 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:15.195911 14313 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:15.196244 14305 server_base.cc:1034] running on GCE node
I20260806 22:47:15.196476 14305 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:47:15.196682 14305 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:47:15.197820 14305 hybrid_clock.cc:648] HybridClock initialized: now 1786056435197801 us; error 37 us; skew 500 ppm
Aug 06 22:47:15 dist-test-slave-zh2d krb5kdc[14148](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056435, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:47:15.200356 14305 init.cc:377] Logged in from keytab as kudu/127.6.40.65@KRBTEST.COM (short username kudu)
I20260806 22:47:15.200827 14305 webserver.cc:460] Webserver started at http://127.6.40.65:37967/ using document root <none> and password file <none>
I20260806 22:47:15.201104 14305 fs_manager.cc:359] Metadata directory not provided
I20260806 22:47:15.201148 14305 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:47:15.201242 14305 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:47:15.202276 14305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/minicluster-data/ts-0/data/instance:
uuid: "d61feeeefed249c7a9a32395aa5b8410"
format_stamp: "Formatted at 2026-08-06 22:47:15 on dist-test-slave-zh2d"
server_key: "e2905424fe674e7bcc97a638ef154966"
server_key_iv: "f86fd25e0f5b996702f7b7340ff2f2ca"
server_key_version: "encryptionkey@0"
I20260806 22:47:15.202572 14305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/minicluster-data/ts-0/wal/instance:
uuid: "d61feeeefed249c7a9a32395aa5b8410"
format_stamp: "Formatted at 2026-08-06 22:47:15 on dist-test-slave-zh2d"
server_key: "e2905424fe674e7bcc97a638ef154966"
server_key_iv: "f86fd25e0f5b996702f7b7340ff2f2ca"
server_key_version: "encryptionkey@0"
I20260806 22:47:15.203909 14305 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:47:15.204648 14323 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:15.204830 14305 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:47:15.204993 14305 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/minicluster-data/ts-0/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/minicluster-data/ts-0/wal
uuid: "d61feeeefed249c7a9a32395aa5b8410"
format_stamp: "Formatted at 2026-08-06 22:47:15 on dist-test-slave-zh2d"
server_key: "e2905424fe674e7bcc97a638ef154966"
server_key_iv: "f86fd25e0f5b996702f7b7340ff2f2ca"
server_key_version: "encryptionkey@0"
I20260806 22:47:15.205080 14305 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:47:15.239410 14305 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:47:15.242063 14305 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:47:15.242245 14305 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:47:15.242516 14305 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:47:15.242870 14305 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:47:15.242941 14305 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:15.242987 14305 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:47:15.243033 14305 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:15.250308 14305 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.65:32967
I20260806 22:47:15.250370 14475 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.65:32967 every 8 connection(s)
I20260806 22:47:15.250743 14305 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
I20260806 22:47:15.256639 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 14305
I20260806 22:47:15.256774 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/minicluster-data/ts-0/wal/instance
I20260806 22:47:15.257022 6305 external_mini_cluster.cc:1428] Setting key c8ba7e0ed44d6451e6bd8c12c53f634c
Aug 06 22:47:15 dist-test-slave-zh2d krb5kdc[14148](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056435, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.65@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
I20260806 22:47:15.263686 14275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:15.251621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.65:43201 (local address 127.6.40.126:34529)
0806 22:47:15.252965 (+ 1344us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:15.252968 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:15.252982 (+ 14us) server_negotiation.cc:407] Connection header received
0806 22:47:15.253534 (+ 552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:15.253535 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:15.253545 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:15.253641 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:15.254680 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:15.255646 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:15.257538 (+ 1892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:15.257835 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:15.257912 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:15.260662 (+ 2750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:15.260664 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:15.260664 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:15.260671 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:15.262201 (+ 1530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:15.262644 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:15.262646 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:15.262646 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:15.262717 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:15.262994 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:15.262995 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:15.262995 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:15.263122 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:15.263195 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:15.263491 (+ 296us) server_negotiation.cc:299] Negotiation successful
0806 22:47:15.263592 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1290}
I20260806 22:47:15.263882 14478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:15.251580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:34529 (local address 127.6.40.65:43201)
0806 22:47:15.251833 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:15.251910 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:15.253428 (+ 1518us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:15.253738 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:15.253743 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:15.254188 (+ 445us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:15.254536 (+ 348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:15.254543 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:15.255782 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:15.255783 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:15.257411 (+ 1628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:15.257415 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:15.257941 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:15.257942 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:15.258120 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:15.258734 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:15.258743 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:15.260573 (+ 1830us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:47:15.262307 (+ 1734us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:15.262315 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:15.262318 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:15.262559 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:15.262797 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:15.262798 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:15.262799 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:15.262915 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:15.263198 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:47:15.263206 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:47:15.263349 (+ 143us) client_negotiation.cc:769] Sending connection context
0806 22:47:15.263460 (+ 111us) client_negotiation.cc:240] Negotiation successful
0806 22:47:15.263629 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":86,"threads_started":1}
I20260806 22:47:15.264490 14476 heartbeater.cc:347] Connected to a master server at 127.6.40.126:34529
I20260806 22:47:15.264611 14476 heartbeater.cc:464] Registering TS with master...
I20260806 22:47:15.264792 14476 heartbeater.cc:511] Master 127.6.40.126:34529 requested a full tablet report, sending...
I20260806 22:47:15.265381 14221 ts_manager.cc:194] Registered new tserver with Master: d61feeeefed249c7a9a32395aa5b8410 (127.6.40.65:32967)
I20260806 22:47:15.266013 14221 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.40.65@KRBTEST.COM'} at 127.6.40.65:43201
WARNING: no policy specified for kudu/127.6.40.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.40.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.40.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.66.keytab.
Entry for principal kudu/127.6.40.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.66.keytab.
I20260806 22:47:15.288601 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.66:0
--local_ip_for_outbound_sockets=127.6.40.66
--webserver_interface=127.6.40.66
--webserver_port=0
--tserver_master_addrs=127.6.40.126:34529
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.66.keytab
--principal=kudu/127.6.40.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:34913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:47:15.372782 14481 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:47:15.373071 14481 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:47:15.373121 14481 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:47:15.374305 14481 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:47:15.374377 14481 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:47:15.374418 14481 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.66
I20260806 22:47:15.375931 14481 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:34913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.66.keytab
--principal=kudu/127.6.40.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.40.66
--webserver_port=0
--tserver_master_addrs=127.6.40.126:34529
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14481
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.66
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:47:15.376209 14481 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:47:15.376447 14481 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:47:15.381620 14490 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:15.381613 14492 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:15.381698 14489 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:15.382005 14481 server_base.cc:1034] running on GCE node
I20260806 22:47:15.382252 14481 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:47:15.382519 14481 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:47:15.383657 14481 hybrid_clock.cc:648] HybridClock initialized: now 1786056435383636 us; error 39 us; skew 500 ppm
Aug 06 22:47:15 dist-test-slave-zh2d krb5kdc[14148](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056435, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:47:15.386240 14481 init.cc:377] Logged in from keytab as kudu/127.6.40.66@KRBTEST.COM (short username kudu)
I20260806 22:47:15.386688 14481 webserver.cc:460] Webserver started at http://127.6.40.66:35769/ using document root <none> and password file <none>
I20260806 22:47:15.386922 14481 fs_manager.cc:359] Metadata directory not provided
I20260806 22:47:15.386991 14481 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:47:15.387144 14481 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:47:15.388218 14481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/minicluster-data/ts-1/data/instance:
uuid: "b7982a2f73204c18b73ef5737cc88885"
format_stamp: "Formatted at 2026-08-06 22:47:15 on dist-test-slave-zh2d"
server_key: "d492ac86e24a9fc4a179a7e73e57e10c"
server_key_iv: "9bc93e26c87e2aa20731dba55627d356"
server_key_version: "encryptionkey@0"
I20260806 22:47:15.388582 14481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/minicluster-data/ts-1/wal/instance:
uuid: "b7982a2f73204c18b73ef5737cc88885"
format_stamp: "Formatted at 2026-08-06 22:47:15 on dist-test-slave-zh2d"
server_key: "d492ac86e24a9fc4a179a7e73e57e10c"
server_key_iv: "9bc93e26c87e2aa20731dba55627d356"
server_key_version: "encryptionkey@0"
I20260806 22:47:15.390100 14481 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:47:15.390899 14502 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:15.391093 14481 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:47:15.391192 14481 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/minicluster-data/ts-1/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/minicluster-data/ts-1/wal
uuid: "b7982a2f73204c18b73ef5737cc88885"
format_stamp: "Formatted at 2026-08-06 22:47:15 on dist-test-slave-zh2d"
server_key: "d492ac86e24a9fc4a179a7e73e57e10c"
server_key_iv: "9bc93e26c87e2aa20731dba55627d356"
server_key_version: "encryptionkey@0"
I20260806 22:47:15.391279 14481 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:47:15.409787 14481 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:47:15.412204 14481 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:47:15.412371 14481 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:47:15.412626 14481 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:47:15.413004 14481 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:47:15.413079 14481 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:15.413120 14481 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:47:15.413147 14481 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:15.419803 14481 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.66:46711
I20260806 22:47:15.419852 14659 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.66:46711 every 8 connection(s)
I20260806 22:47:15.420167 14481 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
I20260806 22:47:15.424157 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 14481
I20260806 22:47:15.424297 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/minicluster-data/ts-1/wal/instance
I20260806 22:47:15.424499 6305 external_mini_cluster.cc:1428] Setting key feb886acc860b5ee8b538dcd147dcb26
Aug 06 22:47:15 dist-test-slave-zh2d krb5kdc[14148](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056435, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.66@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
I20260806 22:47:15.430652 14275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:15.420924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.66:36765 (local address 127.6.40.126:34529)
0806 22:47:15.420973 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:15.420975 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:15.421220 (+ 245us) server_negotiation.cc:407] Connection header received
0806 22:47:15.422234 (+ 1014us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:15.422235 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:15.422248 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:15.422296 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:15.422937 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:15.423491 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:15.424608 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:15.424930 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:15.425001 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:15.427540 (+ 2539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:15.427542 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:15.427542 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:15.427548 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:15.429075 (+ 1527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:15.429611 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:15.429612 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:15.429612 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:15.429664 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:15.429954 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:15.429954 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:15.429954 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:15.430055 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:15.430107 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:15.430499 (+ 392us) server_negotiation.cc:299] Negotiation successful
0806 22:47:15.430581 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 22:47:15.430934 14662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:15.420904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:34529 (local address 127.6.40.66:36765)
0806 22:47:15.421096 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:15.421134 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:15.422108 (+ 974us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:15.422386 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:15.422389 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:15.422642 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:15.422840 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:15.422844 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:15.423603 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:15.423609 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:15.424520 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:15.424523 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:15.425018 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:15.425019 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:15.425169 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:15.425658 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:15.425668 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:15.427418 (+ 1750us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:47:15.429161 (+ 1743us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:15.429170 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:15.429173 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:15.429521 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:15.429733 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:15.429734 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:15.429734 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:15.429882 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:15.430126 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:47:15.430134 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:47:15.430312 (+ 178us) client_negotiation.cc:769] Sending connection context
0806 22:47:15.430442 (+ 130us) client_negotiation.cc:240] Negotiation successful
0806 22:47:15.430648 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":100,"threads_started":1}
I20260806 22:47:15.431496 14660 heartbeater.cc:347] Connected to a master server at 127.6.40.126:34529
I20260806 22:47:15.431602 14660 heartbeater.cc:464] Registering TS with master...
I20260806 22:47:15.431820 14660 heartbeater.cc:511] Master 127.6.40.126:34529 requested a full tablet report, sending...
I20260806 22:47:15.432229 14221 ts_manager.cc:194] Registered new tserver with Master: b7982a2f73204c18b73ef5737cc88885 (127.6.40.66:46711)
I20260806 22:47:15.432611 14221 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.40.66@KRBTEST.COM'} at 127.6.40.66:36765
WARNING: no policy specified for kudu/127.6.40.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.40.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.40.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.67.keytab.
Entry for principal kudu/127.6.40.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.67.keytab.
I20260806 22:47:15.453171 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.67:0
--local_ip_for_outbound_sockets=127.6.40.67
--webserver_interface=127.6.40.67
--webserver_port=0
--tserver_master_addrs=127.6.40.126:34529
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.67.keytab
--principal=kudu/127.6.40.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:34913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:47:15.539121 14668 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:47:15.539357 14668 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:47:15.539399 14668 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:47:15.540673 14668 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:47:15.540740 14668 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:47:15.540786 14668 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.67
I20260806 22:47:15.542299 14668 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:34913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.67.keytab
--principal=kudu/127.6.40.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.40.67
--webserver_port=0
--tserver_master_addrs=127.6.40.126:34529
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14668
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.67
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:47:15.542580 14668 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:47:15.542814 14668 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:47:15.548099 14678 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:15.548116 14682 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:15.548170 14668 server_base.cc:1034] running on GCE node
W20260806 22:47:15.548355 14679 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:15.548575 14668 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:47:15.548830 14668 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:47:15.550040 14668 hybrid_clock.cc:648] HybridClock initialized: now 1786056435550032 us; error 35 us; skew 500 ppm
Aug 06 22:47:15 dist-test-slave-zh2d krb5kdc[14148](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056435, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:47:15.552634 14668 init.cc:377] Logged in from keytab as kudu/127.6.40.67@KRBTEST.COM (short username kudu)
I20260806 22:47:15.553171 14668 webserver.cc:460] Webserver started at http://127.6.40.67:39655/ using document root <none> and password file <none>
I20260806 22:47:15.553434 14668 fs_manager.cc:359] Metadata directory not provided
I20260806 22:47:15.553504 14668 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:47:15.553643 14668 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:47:15.554685 14668 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/minicluster-data/ts-2/data/instance:
uuid: "5c772efa73a04ff392d6ba400acb26b6"
format_stamp: "Formatted at 2026-08-06 22:47:15 on dist-test-slave-zh2d"
server_key: "5cf250ae77dc2a83f26bab35cb224a31"
server_key_iv: "3648c5f0834ed590db27376b76796d43"
server_key_version: "encryptionkey@0"
I20260806 22:47:15.555068 14668 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/minicluster-data/ts-2/wal/instance:
uuid: "5c772efa73a04ff392d6ba400acb26b6"
format_stamp: "Formatted at 2026-08-06 22:47:15 on dist-test-slave-zh2d"
server_key: "5cf250ae77dc2a83f26bab35cb224a31"
server_key_iv: "3648c5f0834ed590db27376b76796d43"
server_key_version: "encryptionkey@0"
I20260806 22:47:15.556445 14668 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:47:15.557386 14691 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:15.557574 14668 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 22:47:15.557696 14668 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/minicluster-data/ts-2/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/minicluster-data/ts-2/wal
uuid: "5c772efa73a04ff392d6ba400acb26b6"
format_stamp: "Formatted at 2026-08-06 22:47:15 on dist-test-slave-zh2d"
server_key: "5cf250ae77dc2a83f26bab35cb224a31"
server_key_iv: "3648c5f0834ed590db27376b76796d43"
server_key_version: "encryptionkey@0"
I20260806 22:47:15.557780 14668 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:47:15.572768 14668 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:47:15.575440 14668 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:47:15.575626 14668 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:47:15.575904 14668 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:47:15.576247 14668 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:47:15.576310 14668 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:15.576352 14668 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:47:15.576383 14668 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:15.583212 14668 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.67:37127
I20260806 22:47:15.583251 14852 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.67:37127 every 8 connection(s)
I20260806 22:47:15.583611 14668 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
I20260806 22:47:15.588994 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 14668
I20260806 22:47:15.589126 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/minicluster-data/ts-2/wal/instance
I20260806 22:47:15.589359 6305 external_mini_cluster.cc:1428] Setting key 76d87a845df600a9d841811fe108601b
Aug 06 22:47:15 dist-test-slave-zh2d krb5kdc[14148](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056435, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.67@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
I20260806 22:47:15.593586 14275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:15.584431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.67:44073 (local address 127.6.40.126:34529)
0806 22:47:15.584495 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:15.584497 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:15.584752 (+ 255us) server_negotiation.cc:407] Connection header received
0806 22:47:15.585809 (+ 1057us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:15.585811 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:15.585825 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:15.585874 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:15.586518 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:15.587026 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:15.588034 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:15.588236 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:15.588314 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:15.590670 (+ 2356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:15.590672 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:15.590673 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:15.590681 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:15.592088 (+ 1407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:15.592570 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:15.592571 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:15.592572 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:15.592630 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:15.592926 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:15.592927 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:15.592927 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:15.593060 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:15.593136 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:15.593413 (+ 277us) server_negotiation.cc:299] Negotiation successful
0806 22:47:15.593494 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 22:47:15.593786 14856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:15.584377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:34529 (local address 127.6.40.67:44073)
0806 22:47:15.584591 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:15.584650 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:15.585688 (+ 1038us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:15.585959 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:15.585963 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:15.586226 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:15.586411 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:15.586415 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:15.587131 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:15.587132 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:15.587941 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:15.587943 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:15.588323 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:15.588324 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:15.588448 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:15.589067 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:15.589075 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:15.590572 (+ 1497us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:47:15.592235 (+ 1663us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:15.592242 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:15.592244 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:15.592475 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:15.592699 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:15.592700 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:15.592700 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:15.592811 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:15.593138 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:47:15.593146 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:47:15.593280 (+ 134us) client_negotiation.cc:769] Sending connection context
0806 22:47:15.593378 (+ 98us) client_negotiation.cc:240] Negotiation successful
0806 22:47:15.593544 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":115,"threads_started":1}
I20260806 22:47:15.594359 14853 heartbeater.cc:347] Connected to a master server at 127.6.40.126:34529
I20260806 22:47:15.594475 14853 heartbeater.cc:464] Registering TS with master...
I20260806 22:47:15.594656 14853 heartbeater.cc:511] Master 127.6.40.126:34529 requested a full tablet report, sending...
I20260806 22:47:15.595044 14221 ts_manager.cc:194] Registered new tserver with Master: 5c772efa73a04ff392d6ba400acb26b6 (127.6.40.67:37127)
I20260806 22:47:15.595506 14221 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.40.67@KRBTEST.COM'} at 127.6.40.67:44073
I20260806 22:47:15.601807 6305 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/krb5kdc/test-admin.keytab.
Aug 06 22:47:15 dist-test-slave-zh2d krb5kdc[14148](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056435, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:47:15.618276 6305 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260806 22:47:15.620079 14864 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:47:15.619216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55326 (local address 127.6.40.65:32967)
0806 22:47:15.619479 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:15.619483 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:15.619494 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:47:15.619541 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:15.619542 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:15.619558 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:15.619631 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:15.619988 (+ 357us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55326: BlockingRecv error: recv got EOF from 127.0.0.1:55326 (error 108)
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":114,"threads_started":1}
W20260806 22:47:15.620198 6305 init.cc:244] Error while opening credential cache '/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/krb5kdc/krb5cc)
Aug 06 22:47:15 dist-test-slave-zh2d krb5kdc[14148](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056435, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:47:15.621255 6305 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 22:47:15 dist-test-slave-zh2d krb5kdc[14148](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056435, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:47:15.622138 6305 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 22:47:15 dist-test-slave-zh2d krb5kdc[14148](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056435, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.40.65@KRBTEST.COM
I20260806 22:47:15.630208 14864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:15.622661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55334 (local address 127.6.40.65:32967)
0806 22:47:15.622723 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:15.622725 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:15.622780 (+ 55us) server_negotiation.cc:407] Connection header received
0806 22:47:15.622868 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:15.622869 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:15.622880 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:15.622954 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:15.623346 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:15.624088 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:15.624822 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:15.625169 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:15.625252 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:15.627314 (+ 2062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:15.627323 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:15.627324 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:15.627331 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:15.628977 (+ 1646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:15.629361 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:15.629364 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:15.629365 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:15.629428 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:15.629633 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:15.629634 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:15.629635 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:15.629770 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:15.629918 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:15.630013 (+ 95us) server_negotiation.cc:299] Negotiation successful
0806 22:47:15.630093 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
W20260806 22:47:15.632064 14864 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:47:15.631564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55344 (local address 127.6.40.65:32967)
0806 22:47:15.631598 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:15.631599 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:15.631635 (+ 36us) server_negotiation.cc:407] Connection header received
0806 22:47:15.631681 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:15.631681 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:15.631689 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:15.631734 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:15.632002 (+ 268us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55344: BlockingRecv error: recv got EOF from 127.0.0.1:55344 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260806 22:47:15.632192 6305 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/krb5kdc/krb5cc' of type FILE
Aug 06 22:47:15 dist-test-slave-zh2d krb5kdc[14148](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056435, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:47:15.633169 6305 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 22:47:15 dist-test-slave-zh2d krb5kdc[14148](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056435, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:47:15.634130 6305 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 22:47:15 dist-test-slave-zh2d krb5kdc[14148](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056435, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.40.65@KRBTEST.COM
I20260806 22:47:15.641309 14864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:15.634629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55360 (local address 127.6.40.65:32967)
0806 22:47:15.634675 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:15.634677 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:15.634728 (+ 51us) server_negotiation.cc:407] Connection header received
0806 22:47:15.634771 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:15.634771 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:15.634780 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:15.634828 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:15.635179 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:15.635657 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:15.636392 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:15.636597 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:15.636648 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:15.638940 (+ 2292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:15.638942 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:15.638942 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:15.638948 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:15.640339 (+ 1391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:15.640653 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:15.640653 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:15.640654 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:15.640700 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:15.640930 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:15.640931 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:15.640931 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:15.641028 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:15.641102 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:15.641193 (+ 91us) server_negotiation.cc:299] Negotiation successful
0806 22:47:15.641249 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
W20260806 22:47:15.642897 14864 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:47:15.642318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55366 (local address 127.6.40.65:32967)
0806 22:47:15.642380 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:15.642382 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:15.642459 (+ 77us) server_negotiation.cc:407] Connection header received
0806 22:47:15.642526 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:15.642527 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:15.642536 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:15.642590 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:15.642850 (+ 260us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55366: BlockingRecv error: recv got EOF from 127.0.0.1:55366 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260806 22:47:15.642985 6305 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056405724609-6305-0/krb5kdc/krb5cc' of type FILE
Aug 06 22:47:15 dist-test-slave-zh2d krb5kdc[14148](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056435, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:47:15.643919 6305 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 22:47:15 dist-test-slave-zh2d krb5kdc[14148](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056435, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:47:15.644734 6305 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 22:47:15 dist-test-slave-zh2d krb5kdc[14148](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056435, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.40.65@KRBTEST.COM
I20260806 22:47:15.651257 14864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:15.645336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55382 (local address 127.6.40.65:32967)
0806 22:47:15.645379 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:15.645381 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:15.645450 (+ 69us) server_negotiation.cc:407] Connection header received
0806 22:47:15.645485 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:15.645486 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:15.645494 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:15.645538 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:15.645891 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:15.646360 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:15.646983 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:15.647154 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:15.647202 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:15.649158 (+ 1956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:15.649159 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:15.649159 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:15.649164 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:15.650279 (+ 1115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:15.650622 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:15.650623 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:15.650623 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:15.650672 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:15.650836 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:15.650837 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:15.650837 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:15.650929 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:15.650998 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:15.651098 (+ 100us) server_negotiation.cc:299] Negotiation successful
0806 22:47:15.651149 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 22:47:15.651927 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 14305
I20260806 22:47:15.664889 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 14481
I20260806 22:47:15.678642 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 14668
I20260806 22:47:15.687922 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 14165
2026-08-06T22:47:15Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (3453 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:47:15 dist-test-slave-zh2d krb5kdc[14880](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:47:15 dist-test-slave-zh2d krb5kdc[14880](info): set up 2 sockets
Aug 06 22:47:15 dist-test-slave-zh2d krb5kdc[14880](info): commencing operation
krb5kdc: starting...
W20260806 22:47:16.969036 6305 mini_kdc.cc:121] Time spent starting KDC: real 1.256s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/krb5kdc/test-user.keytab.
Aug 06 22:47:17 dist-test-slave-zh2d krb5kdc[14880](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056437, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:47:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:47:17Z Disabled control of system clock
WARNING: no policy specified for oryx/127.6.40.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.40.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.40.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/krb5kdc/oryx_127.6.40.126.keytab.
Entry for principal oryx/127.6.40.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/krb5kdc/oryx_127.6.40.126.keytab.
I20260806 22:47:17.096300 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.126:38077
--webserver_interface=127.6.40.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/krb5kdc/oryx_127.6.40.126.keytab
--principal=oryx/127.6.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:34373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.40.126:38077
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:47:17.181182 14899 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:47:17.181443 14899 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:47:17.181481 14899 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:47:17.182629 14899 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:47:17.182698 14899 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:47:17.182721 14899 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:47:17.182740 14899 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:47:17.182767 14899 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:47:17.184276 14899 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:34373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.40.126:38077
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/krb5kdc/oryx_127.6.40.126.keytab
--principal=oryx/127.6.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.126:38077
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14899
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:47:17.184564 14899 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:47:17.184805 14899 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:47:17.189913 14908 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:17.190012 14906 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:17.190112 14899 server_base.cc:1034] running on GCE node
W20260806 22:47:17.189922 14905 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:17.190493 14899 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:47:17.190747 14899 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:47:17.191913 14899 hybrid_clock.cc:648] HybridClock initialized: now 1786056437191896 us; error 34 us; skew 500 ppm
Aug 06 22:47:17 dist-test-slave-zh2d krb5kdc[14880](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056437, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:47:17.194464 14899 init.cc:377] Logged in from keytab as oryx/127.6.40.126@KRBTEST.COM (short username oryx)
I20260806 22:47:17.194929 14899 webserver.cc:460] Webserver started at http://127.6.40.126:33657/ using document root <none> and password file <none>
I20260806 22:47:17.195127 14899 fs_manager.cc:359] Metadata directory not provided
I20260806 22:47:17.195166 14899 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:47:17.195257 14899 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:47:17.196242 14899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/master-0/data/instance:
uuid: "eb7a8590eeb54daab448b3b04d3dfacf"
format_stamp: "Formatted at 2026-08-06 22:47:17 on dist-test-slave-zh2d"
server_key: "3314e0fbbe8b9119c7daf0b6ed6c3b8e"
server_key_iv: "a7759d287b36145b3555ada2e570ada0"
server_key_version: "encryptionkey@0"
I20260806 22:47:17.196547 14899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/master-0/wal/instance:
uuid: "eb7a8590eeb54daab448b3b04d3dfacf"
format_stamp: "Formatted at 2026-08-06 22:47:17 on dist-test-slave-zh2d"
server_key: "3314e0fbbe8b9119c7daf0b6ed6c3b8e"
server_key_iv: "a7759d287b36145b3555ada2e570ada0"
server_key_version: "encryptionkey@0"
I20260806 22:47:17.197937 14899 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:47:17.198678 14918 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:17.198861 14899 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:47:17.198949 14899 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/master-0/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/master-0/wal
uuid: "eb7a8590eeb54daab448b3b04d3dfacf"
format_stamp: "Formatted at 2026-08-06 22:47:17 on dist-test-slave-zh2d"
server_key: "3314e0fbbe8b9119c7daf0b6ed6c3b8e"
server_key_iv: "a7759d287b36145b3555ada2e570ada0"
server_key_version: "encryptionkey@0"
I20260806 22:47:17.199003 14899 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:47:17.223902 14899 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:47:17.226379 14899 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:47:17.226501 14899 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:47:17.230736 14899 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.126:38077
I20260806 22:47:17.230796 14989 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.126:38077 every 8 connection(s)
I20260806 22:47:17.231190 14899 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
I20260806 22:47:17.231774 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 14899
I20260806 22:47:17.231869 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/master-0/wal/instance
I20260806 22:47:17.232090 6305 external_mini_cluster.cc:1428] Setting key 193ecad194a1bb33edf0da9cc74611a4
I20260806 22:47:17.232457 14990 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:47:17.236054 14990 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb7a8590eeb54daab448b3b04d3dfacf: Bootstrap starting.
I20260806 22:47:17.236753 14990 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eb7a8590eeb54daab448b3b04d3dfacf: Neither blocks nor log segments found. Creating new log.
I20260806 22:47:17.237116 14990 log.cc:824] T 00000000000000000000000000000000 P eb7a8590eeb54daab448b3b04d3dfacf: Log is configured to *not* fsync() on all Append() calls
I20260806 22:47:17.238042 14990 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb7a8590eeb54daab448b3b04d3dfacf: No bootstrap required, opened a new log
Aug 06 22:47:17 dist-test-slave-zh2d krb5kdc[14880](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056437, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM
I20260806 22:47:17.239933 14990 raft_consensus.cc:348] T 00000000000000000000000000000000 P eb7a8590eeb54daab448b3b04d3dfacf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb7a8590eeb54daab448b3b04d3dfacf" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 38077 } }
I20260806 22:47:17.240139 14990 raft_consensus.cc:374] T 00000000000000000000000000000000 P eb7a8590eeb54daab448b3b04d3dfacf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:47:17.240182 14990 raft_consensus.cc:729] T 00000000000000000000000000000000 P eb7a8590eeb54daab448b3b04d3dfacf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb7a8590eeb54daab448b3b04d3dfacf, State: Initialized, Role: FOLLOWER
I20260806 22:47:17.240317 14990 consensus_queue.cc:260] T 00000000000000000000000000000000 P eb7a8590eeb54daab448b3b04d3dfacf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb7a8590eeb54daab448b3b04d3dfacf" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 38077 } }
I20260806 22:47:17.240427 14990 raft_consensus.cc:388] T 00000000000000000000000000000000 P eb7a8590eeb54daab448b3b04d3dfacf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:47:17.240460 14990 raft_consensus.cc:482] T 00000000000000000000000000000000 P eb7a8590eeb54daab448b3b04d3dfacf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:47:17.240521 14990 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eb7a8590eeb54daab448b3b04d3dfacf [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:47:17.241333 14990 raft_consensus.cc:504] T 00000000000000000000000000000000 P eb7a8590eeb54daab448b3b04d3dfacf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb7a8590eeb54daab448b3b04d3dfacf" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 38077 } }
I20260806 22:47:17.241451 14990 leader_election.cc:302] T 00000000000000000000000000000000 P eb7a8590eeb54daab448b3b04d3dfacf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eb7a8590eeb54daab448b3b04d3dfacf; no voters:
I20260806 22:47:17.241631 14990 leader_election.cc:288] T 00000000000000000000000000000000 P eb7a8590eeb54daab448b3b04d3dfacf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:47:17.241744 14999 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eb7a8590eeb54daab448b3b04d3dfacf [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:47:17.241919 14999 raft_consensus.cc:686] T 00000000000000000000000000000000 P eb7a8590eeb54daab448b3b04d3dfacf [term 1 LEADER]: Becoming Leader. State: Replica: eb7a8590eeb54daab448b3b04d3dfacf, State: Running, Role: LEADER
I20260806 22:47:17.241967 14990 sys_catalog.cc:564] T 00000000000000000000000000000000 P eb7a8590eeb54daab448b3b04d3dfacf [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:47:17.242707 14999 consensus_queue.cc:237] T 00000000000000000000000000000000 P eb7a8590eeb54daab448b3b04d3dfacf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb7a8590eeb54daab448b3b04d3dfacf" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 38077 } }
I20260806 22:47:17.243346 15000 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb7a8590eeb54daab448b3b04d3dfacf [sys.catalog]: SysCatalogTable state changed. Reason: New leader eb7a8590eeb54daab448b3b04d3dfacf. Latest consensus state: current_term: 1 leader_uuid: "eb7a8590eeb54daab448b3b04d3dfacf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb7a8590eeb54daab448b3b04d3dfacf" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 38077 } } }
I20260806 22:47:17.243429 15000 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb7a8590eeb54daab448b3b04d3dfacf [sys.catalog]: This master's current role is: LEADER
I20260806 22:47:17.243702 15001 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb7a8590eeb54daab448b3b04d3dfacf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eb7a8590eeb54daab448b3b04d3dfacf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb7a8590eeb54daab448b3b04d3dfacf" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 38077 } } }
I20260806 22:47:17.243781 15001 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb7a8590eeb54daab448b3b04d3dfacf [sys.catalog]: This master's current role is: LEADER
I20260806 22:47:17.243772 15018 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:47:17.244407 14994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:17.232657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48052 (local address 127.6.40.126:38077)
0806 22:47:17.232878 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:17.232888 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:17.232924 (+ 36us) server_negotiation.cc:407] Connection header received
0806 22:47:17.233533 (+ 609us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:17.233543 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:17.233608 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:17.233764 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:17.234398 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:17.235765 (+ 1367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:17.236636 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:17.237285 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:17.237399 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:17.240032 (+ 2633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:17.240036 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:17.240036 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:17.240048 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:17.242236 (+ 2188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:17.242700 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:17.242704 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:17.242705 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:17.242858 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:17.243152 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:17.243153 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:17.243153 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:17.243565 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:17.244092 (+ 527us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:17.244161 (+ 69us) server_negotiation.cc:299] Negotiation successful
0806 22:47:17.244261 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":100,"threads_started":1}
I20260806 22:47:17.244493 15018 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:47:17.246031 15018 catalog_manager.cc:1347] Generated new cluster ID: 16746f209dcb48eb906c4d27294225e8
I20260806 22:47:17.246088 15018 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:47:17.262744 15018 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:47:17.263428 15018 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:47:17.271194 15018 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eb7a8590eeb54daab448b3b04d3dfacf: Generated new TSK 0
I20260806 22:47:17.271378 15018 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.6.40.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.40.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.40.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/krb5kdc/oryx_127.6.40.65.keytab.
Entry for principal oryx/127.6.40.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/krb5kdc/oryx_127.6.40.65.keytab.
I20260806 22:47:17.304003 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.65:0
--local_ip_for_outbound_sockets=127.6.40.65
--webserver_interface=127.6.40.65
--webserver_port=0
--tserver_master_addrs=127.6.40.126:38077
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/krb5kdc/oryx_127.6.40.65.keytab
--principal=oryx/127.6.40.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:34373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:47:17.388491 15024 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:47:17.388718 15024 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:47:17.388761 15024 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:47:17.390046 15024 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:47:17.390117 15024 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:47:17.390161 15024 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.65
I20260806 22:47:17.391647 15024 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:34373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/krb5kdc/oryx_127.6.40.65.keytab
--principal=oryx/127.6.40.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.40.65
--webserver_port=0
--tserver_master_addrs=127.6.40.126:38077
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15024
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.65
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:47:17.391917 15024 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:47:17.392153 15024 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:47:17.397354 15032 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:17.397418 15024 server_base.cc:1034] running on GCE node
W20260806 22:47:17.397336 15031 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:17.397338 15034 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:17.397749 15024 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:47:17.398067 15024 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:47:17.399228 15024 hybrid_clock.cc:648] HybridClock initialized: now 1786056437399204 us; error 35 us; skew 500 ppm
Aug 06 22:47:17 dist-test-slave-zh2d krb5kdc[14880](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056437, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.40.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:47:17.401855 15024 init.cc:377] Logged in from keytab as oryx/127.6.40.65@KRBTEST.COM (short username oryx)
I20260806 22:47:17.402333 15024 webserver.cc:460] Webserver started at http://127.6.40.65:38213/ using document root <none> and password file <none>
I20260806 22:47:17.402566 15024 fs_manager.cc:359] Metadata directory not provided
I20260806 22:47:17.402634 15024 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:47:17.402767 15024 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:47:17.403808 15024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/ts-0/data/instance:
uuid: "35a7f06767bb4b81999ee9fa87d12aa3"
format_stamp: "Formatted at 2026-08-06 22:47:17 on dist-test-slave-zh2d"
server_key: "31dd1440e2bd4eac04444def40941b32"
server_key_iv: "aa3e0aa194adcba88785fa374dd7c6f0"
server_key_version: "encryptionkey@0"
I20260806 22:47:17.404175 15024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/ts-0/wal/instance:
uuid: "35a7f06767bb4b81999ee9fa87d12aa3"
format_stamp: "Formatted at 2026-08-06 22:47:17 on dist-test-slave-zh2d"
server_key: "31dd1440e2bd4eac04444def40941b32"
server_key_iv: "aa3e0aa194adcba88785fa374dd7c6f0"
server_key_version: "encryptionkey@0"
I20260806 22:47:17.405613 15024 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:47:17.406375 15044 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:17.406569 15024 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:47:17.406667 15024 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/ts-0/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/ts-0/wal
uuid: "35a7f06767bb4b81999ee9fa87d12aa3"
format_stamp: "Formatted at 2026-08-06 22:47:17 on dist-test-slave-zh2d"
server_key: "31dd1440e2bd4eac04444def40941b32"
server_key_iv: "aa3e0aa194adcba88785fa374dd7c6f0"
server_key_version: "encryptionkey@0"
I20260806 22:47:17.406749 15024 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:47:17.414548 15024 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:47:17.416333 15024 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:47:17.416505 15024 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:47:17.416724 15024 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:47:17.417069 15024 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:47:17.417124 15024 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:17.417166 15024 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:47:17.417193 15024 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:17.423224 15024 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.65:43189
I20260806 22:47:17.423275 15205 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.65:43189 every 8 connection(s)
I20260806 22:47:17.423586 15024 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
I20260806 22:47:17.429559 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 15024
I20260806 22:47:17.429651 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/ts-0/wal/instance
I20260806 22:47:17.429886 6305 external_mini_cluster.cc:1428] Setting key 1bf73e6ac89764862e6e67c56abe3118
Aug 06 22:47:17 dist-test-slave-zh2d krb5kdc[14880](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056437, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.40.65@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM
I20260806 22:47:17.433619 14994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:17.424261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.65:60325 (local address 127.6.40.126:38077)
0806 22:47:17.424308 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:17.424310 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:17.424581 (+ 271us) server_negotiation.cc:407] Connection header received
0806 22:47:17.425217 (+ 636us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:17.425219 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:17.425233 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:17.425284 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:17.426249 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:17.426766 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:17.427779 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:17.427992 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:17.428069 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:17.430639 (+ 2570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:17.430641 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:17.430642 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:17.430649 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:17.432106 (+ 1457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:17.432566 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:17.432567 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:17.432568 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:17.432635 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:17.432937 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:17.432938 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:17.432938 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:17.433071 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:17.433148 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:17.433444 (+ 296us) server_negotiation.cc:299] Negotiation successful
0806 22:47:17.433530 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 22:47:17.433820 15208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:17.424225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:38077 (local address 127.6.40.65:60325)
0806 22:47:17.424419 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:17.424486 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:17.425101 (+ 615us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:17.425351 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:17.425354 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:17.425958 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:17.426147 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:17.426151 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:17.426873 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:17.426875 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:17.427687 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:17.427689 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:17.428090 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:17.428091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:17.428215 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:17.428794 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:17.428802 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:17.430557 (+ 1755us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:47:17.432215 (+ 1658us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:17.432223 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:17.432226 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:17.432482 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:17.432713 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:17.432714 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:17.432714 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:17.432824 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:17.433151 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:47:17.433159 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:47:17.433299 (+ 140us) client_negotiation.cc:769] Sending connection context
0806 22:47:17.433399 (+ 100us) client_negotiation.cc:240] Negotiation successful
0806 22:47:17.433563 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":93,"threads_started":1}
I20260806 22:47:17.434320 15206 heartbeater.cc:347] Connected to a master server at 127.6.40.126:38077
I20260806 22:47:17.434428 15206 heartbeater.cc:464] Registering TS with master...
I20260806 22:47:17.434616 15206 heartbeater.cc:511] Master 127.6.40.126:38077 requested a full tablet report, sending...
I20260806 22:47:17.435223 14940 ts_manager.cc:194] Registered new tserver with Master: 35a7f06767bb4b81999ee9fa87d12aa3 (127.6.40.65:43189)
I20260806 22:47:17.436198 14940 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.40.65@KRBTEST.COM'} at 127.6.40.65:60325
WARNING: no policy specified for oryx/127.6.40.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.40.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.40.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/krb5kdc/oryx_127.6.40.66.keytab.
Entry for principal oryx/127.6.40.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/krb5kdc/oryx_127.6.40.66.keytab.
I20260806 22:47:17.459880 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.66:0
--local_ip_for_outbound_sockets=127.6.40.66
--webserver_interface=127.6.40.66
--webserver_port=0
--tserver_master_addrs=127.6.40.126:38077
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/krb5kdc/oryx_127.6.40.66.keytab
--principal=oryx/127.6.40.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:34373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:47:17.545419 15211 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:47:17.545614 15211 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:47:17.545639 15211 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:47:17.546710 15211 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:47:17.546744 15211 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:47:17.546769 15211 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.66
I20260806 22:47:17.548132 15211 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:34373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/krb5kdc/oryx_127.6.40.66.keytab
--principal=oryx/127.6.40.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.40.66
--webserver_port=0
--tserver_master_addrs=127.6.40.126:38077
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15211
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.66
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:47:17.548307 15211 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:47:17.548489 15211 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:47:17.553609 15223 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:17.553617 15221 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:17.553645 15211 server_base.cc:1034] running on GCE node
W20260806 22:47:17.553622 15219 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:17.554013 15211 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:47:17.554255 15211 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:47:17.555397 15211 hybrid_clock.cc:648] HybridClock initialized: now 1786056437555384 us; error 40 us; skew 500 ppm
Aug 06 22:47:17 dist-test-slave-zh2d krb5kdc[14880](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056437, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.40.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:47:17.558056 15211 init.cc:377] Logged in from keytab as oryx/127.6.40.66@KRBTEST.COM (short username oryx)
I20260806 22:47:17.558485 15211 webserver.cc:460] Webserver started at http://127.6.40.66:40053/ using document root <none> and password file <none>
I20260806 22:47:17.558701 15211 fs_manager.cc:359] Metadata directory not provided
I20260806 22:47:17.558745 15211 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:47:17.558902 15211 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:47:17.559947 15211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/ts-1/data/instance:
uuid: "d1fec59295474125ae87691d06eaf345"
format_stamp: "Formatted at 2026-08-06 22:47:17 on dist-test-slave-zh2d"
server_key: "376719fcbdf4d4a70eb6cd2e7fa3c5b0"
server_key_iv: "2316a0bb93da27e3bbfc86d97120ff9d"
server_key_version: "encryptionkey@0"
I20260806 22:47:17.560302 15211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/ts-1/wal/instance:
uuid: "d1fec59295474125ae87691d06eaf345"
format_stamp: "Formatted at 2026-08-06 22:47:17 on dist-test-slave-zh2d"
server_key: "376719fcbdf4d4a70eb6cd2e7fa3c5b0"
server_key_iv: "2316a0bb93da27e3bbfc86d97120ff9d"
server_key_version: "encryptionkey@0"
I20260806 22:47:17.561710 15211 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:47:17.562433 15232 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:17.562621 15211 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:47:17.562729 15211 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/ts-1/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/ts-1/wal
uuid: "d1fec59295474125ae87691d06eaf345"
format_stamp: "Formatted at 2026-08-06 22:47:17 on dist-test-slave-zh2d"
server_key: "376719fcbdf4d4a70eb6cd2e7fa3c5b0"
server_key_iv: "2316a0bb93da27e3bbfc86d97120ff9d"
server_key_version: "encryptionkey@0"
I20260806 22:47:17.562804 15211 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:47:17.571636 15211 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:47:17.573961 15211 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:47:17.574120 15211 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:47:17.574340 15211 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:47:17.574641 15211 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:47:17.574694 15211 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:17.574734 15211 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:47:17.574764 15211 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:17.580950 15211 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.66:34829
I20260806 22:47:17.581007 15380 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.66:34829 every 8 connection(s)
I20260806 22:47:17.581295 15211 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
I20260806 22:47:17.585588 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 15211
I20260806 22:47:17.585706 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/ts-1/wal/instance
I20260806 22:47:17.585917 6305 external_mini_cluster.cc:1428] Setting key 1d4d33d697defe8d249ce7045589ef9a
Aug 06 22:47:17 dist-test-slave-zh2d krb5kdc[14880](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056437, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.40.66@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM
I20260806 22:47:17.591137 15385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:17.581941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:38077 (local address 127.6.40.66:52541)
0806 22:47:17.582131 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:17.582166 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:17.582661 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:17.582948 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:17.582951 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:17.583507 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:17.583690 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:17.583694 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:17.584392 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:17.584393 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:17.585296 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:17.585298 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:17.585698 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:17.585699 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:17.585858 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:17.586448 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:17.586458 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:17.587931 (+ 1473us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:47:17.589666 (+ 1735us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:17.589674 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:17.589676 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:17.589933 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:17.590188 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:17.590189 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:17.590190 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:17.590345 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:17.590649 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:47:17.590655 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:47:17.590772 (+ 117us) client_negotiation.cc:769] Sending connection context
0806 22:47:17.590854 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 22:47:17.590983 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":88,"threads_started":1}
I20260806 22:47:17.591136 14994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:17.581963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.66:52541 (local address 127.6.40.126:38077)
0806 22:47:17.582011 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:17.582013 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:17.582250 (+ 237us) server_negotiation.cc:407] Connection header received
0806 22:47:17.582766 (+ 516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:17.582767 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:17.582779 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:17.582847 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:17.583795 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:17.584294 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:17.585389 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:17.585596 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:17.585686 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:17.588030 (+ 2344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:17.588032 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:17.588033 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:17.588040 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:17.589559 (+ 1519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:17.590031 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:17.590032 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:17.590033 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:17.590095 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:17.590428 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:17.590429 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:17.590430 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:17.590579 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:17.590651 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:17.590902 (+ 251us) server_negotiation.cc:299] Negotiation successful
0806 22:47:17.590997 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 22:47:17.591655 15382 heartbeater.cc:347] Connected to a master server at 127.6.40.126:38077
I20260806 22:47:17.591763 15382 heartbeater.cc:464] Registering TS with master...
I20260806 22:47:17.591987 15382 heartbeater.cc:511] Master 127.6.40.126:38077 requested a full tablet report, sending...
I20260806 22:47:17.592386 14940 ts_manager.cc:194] Registered new tserver with Master: d1fec59295474125ae87691d06eaf345 (127.6.40.66:34829)
I20260806 22:47:17.592777 14940 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.40.66@KRBTEST.COM'} at 127.6.40.66:52541
WARNING: no policy specified for oryx/127.6.40.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.40.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.40.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/krb5kdc/oryx_127.6.40.67.keytab.
Entry for principal oryx/127.6.40.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/krb5kdc/oryx_127.6.40.67.keytab.
I20260806 22:47:17.615391 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.67:0
--local_ip_for_outbound_sockets=127.6.40.67
--webserver_interface=127.6.40.67
--webserver_port=0
--tserver_master_addrs=127.6.40.126:38077
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/krb5kdc/oryx_127.6.40.67.keytab
--principal=oryx/127.6.40.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:34373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:47:17.703790 15391 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:47:17.704013 15391 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:47:17.704041 15391 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:47:17.705788 15391 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:47:17.705855 15391 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:47:17.705904 15391 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.67
I20260806 22:47:17.707700 15391 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:34373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/krb5kdc/oryx_127.6.40.67.keytab
--principal=oryx/127.6.40.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.40.67
--webserver_port=0
--tserver_master_addrs=127.6.40.126:38077
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15391
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.67
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:47:17.707917 15391 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:47:17.708115 15391 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:47:17.713974 15398 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:17.713974 15399 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:17.714061 15403 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:17.714339 15391 server_base.cc:1034] running on GCE node
I20260806 22:47:17.714594 15391 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:47:17.714808 15391 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:47:17.715983 15391 hybrid_clock.cc:648] HybridClock initialized: now 1786056437715967 us; error 31 us; skew 500 ppm
Aug 06 22:47:17 dist-test-slave-zh2d krb5kdc[14880](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056437, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.40.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:47:17.718592 15391 init.cc:377] Logged in from keytab as oryx/127.6.40.67@KRBTEST.COM (short username oryx)
I20260806 22:47:17.719048 15391 webserver.cc:460] Webserver started at http://127.6.40.67:43699/ using document root <none> and password file <none>
I20260806 22:47:17.719304 15391 fs_manager.cc:359] Metadata directory not provided
I20260806 22:47:17.719377 15391 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:47:17.719527 15391 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:47:17.720566 15391 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/ts-2/data/instance:
uuid: "62e43f48d5464af9bcbd68d8fb9877d4"
format_stamp: "Formatted at 2026-08-06 22:47:17 on dist-test-slave-zh2d"
server_key: "8be2ae534ae39f5ffd3abbe3698cbfbe"
server_key_iv: "8adfcfcfa2f2154f3ed6bc86235d6dfe"
server_key_version: "encryptionkey@0"
I20260806 22:47:17.720990 15391 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/ts-2/wal/instance:
uuid: "62e43f48d5464af9bcbd68d8fb9877d4"
format_stamp: "Formatted at 2026-08-06 22:47:17 on dist-test-slave-zh2d"
server_key: "8be2ae534ae39f5ffd3abbe3698cbfbe"
server_key_iv: "8adfcfcfa2f2154f3ed6bc86235d6dfe"
server_key_version: "encryptionkey@0"
I20260806 22:47:17.722363 15391 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:47:17.723263 15411 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:17.723500 15391 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 22:47:17.723603 15391 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/ts-2/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/ts-2/wal
uuid: "62e43f48d5464af9bcbd68d8fb9877d4"
format_stamp: "Formatted at 2026-08-06 22:47:17 on dist-test-slave-zh2d"
server_key: "8be2ae534ae39f5ffd3abbe3698cbfbe"
server_key_iv: "8adfcfcfa2f2154f3ed6bc86235d6dfe"
server_key_version: "encryptionkey@0"
I20260806 22:47:17.723687 15391 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:47:17.740684 15391 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:47:17.743405 15391 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:47:17.743587 15391 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:47:17.743930 15391 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:47:17.744218 15391 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:47:17.744249 15391 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:17.744271 15391 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:47:17.744307 15391 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:17.750638 15391 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.67:45455
I20260806 22:47:17.750692 15559 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.67:45455 every 8 connection(s)
I20260806 22:47:17.751011 15391 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
Aug 06 22:47:17 dist-test-slave-zh2d krb5kdc[14880](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056437, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.40.67@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM
I20260806 22:47:17.760130 14994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:17.751752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.67:41551 (local address 127.6.40.126:38077)
0806 22:47:17.751798 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:17.751800 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:17.752004 (+ 204us) server_negotiation.cc:407] Connection header received
0806 22:47:17.752541 (+ 537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:17.752542 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:17.752554 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:17.752601 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:17.753640 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:17.754156 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:17.755151 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:17.755328 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:17.755378 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:17.757529 (+ 2151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:17.757530 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:17.757530 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:17.757537 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:17.758980 (+ 1443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:17.759341 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:17.759342 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:17.759343 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:17.759393 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:17.759604 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:17.759605 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:17.759605 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:17.759705 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:17.759755 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:17.760003 (+ 248us) server_negotiation.cc:299] Negotiation successful
0806 22:47:17.760063 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 22:47:17.760336 15562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:17.751729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:38077 (local address 127.6.40.67:41551)
0806 22:47:17.751890 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:17.751928 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:17.752450 (+ 522us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:17.752664 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:17.752668 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:17.753344 (+ 676us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:17.753551 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:17.753556 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:17.754237 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:17.754238 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:17.755077 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:17.755079 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:17.755423 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:17.755424 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:17.755526 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:17.755968 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:17.755976 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:17.757441 (+ 1465us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:47:17.759049 (+ 1608us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:17.759056 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:17.759058 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:17.759275 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:17.759451 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:17.759452 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:17.759452 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:17.759549 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:17.759759 (+ 210us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:47:17.759763 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:47:17.759877 (+ 114us) client_negotiation.cc:769] Sending connection context
0806 22:47:17.759965 (+ 88us) client_negotiation.cc:240] Negotiation successful
0806 22:47:17.760088 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":94,"threads_started":1}
I20260806 22:47:17.760886 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 15391
I20260806 22:47:17.760941 15560 heartbeater.cc:347] Connected to a master server at 127.6.40.126:38077
I20260806 22:47:17.761022 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/ts-2/wal/instance
I20260806 22:47:17.761039 15560 heartbeater.cc:464] Registering TS with master...
I20260806 22:47:17.761250 6305 external_mini_cluster.cc:1428] Setting key a1c8847960c9b575d71091c943a69594
I20260806 22:47:17.761252 15560 heartbeater.cc:511] Master 127.6.40.126:38077 requested a full tablet report, sending...
I20260806 22:47:17.761718 14938 ts_manager.cc:194] Registered new tserver with Master: 62e43f48d5464af9bcbd68d8fb9877d4 (127.6.40.67:45455)
I20260806 22:47:17.762218 14938 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.40.67@KRBTEST.COM'} at 127.6.40.67:41551
I20260806 22:47:17.762256 6305 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.6.40.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.40.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.40.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/krb5kdc/oryx_127.6.40.125.keytab.
Entry for principal oryx/127.6.40.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/krb5kdc/oryx_127.6.40.125.keytab.
I20260806 22:47:17.791903 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.125:42247
--webserver_interface=127.6.40.125
--webserver_port=0
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/krb5kdc/oryx_127.6.40.125.keytab
--principal=oryx/127.6.40.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:34373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.40.126:38077,127.6.40.125:42247
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:47:17.877440 15567 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:47:17.877671 15567 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:47:17.877694 15567 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:47:17.878830 15567 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:47:17.878867 15567 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:47:17.878881 15567 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:47:17.878895 15567 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:47:17.878907 15567 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:47:17.880357 15567 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:34373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.6.40.126:38077,127.6.40.125:42247
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/krb5kdc/oryx_127.6.40.125.keytab
--principal=oryx/127.6.40.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.125:42247
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.6.40.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15567
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
W20260806 22:47:17.880477 15567 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.6.40.126:38077,127.6.40.125:42247'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260806 22:47:17.880596 15567 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:47:17.880784 15567 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:47:17.885762 15582 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:17.885857 15579 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:17.885762 15580 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:17.886032 15567 server_base.cc:1034] running on GCE node
I20260806 22:47:17.886313 15567 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:47:17.886610 15567 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:47:17.887773 15567 hybrid_clock.cc:648] HybridClock initialized: now 1786056437887754 us; error 35 us; skew 500 ppm
Aug 06 22:47:17 dist-test-slave-zh2d krb5kdc[14880](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056437, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.40.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:47:17.890380 15567 init.cc:377] Logged in from keytab as oryx/127.6.40.125@KRBTEST.COM (short username oryx)
I20260806 22:47:17.890825 15567 webserver.cc:460] Webserver started at http://127.6.40.125:36007/ using document root <none> and password file <none>
I20260806 22:47:17.891067 15567 fs_manager.cc:359] Metadata directory not provided
I20260806 22:47:17.891137 15567 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:47:17.891301 15567 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:47:17.892377 15567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/master-1/data/instance:
uuid: "0016a76e8ad5476995c85a46a1fdcb6b"
format_stamp: "Formatted at 2026-08-06 22:47:17 on dist-test-slave-zh2d"
server_key: "4d0c5499ba484d8a1c4e927f4686ac30"
server_key_iv: "04d18ec6973cff66237f58ef764a3074"
server_key_version: "encryptionkey@0"
I20260806 22:47:17.892756 15567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/master-1/wal/instance:
uuid: "0016a76e8ad5476995c85a46a1fdcb6b"
format_stamp: "Formatted at 2026-08-06 22:47:17 on dist-test-slave-zh2d"
server_key: "4d0c5499ba484d8a1c4e927f4686ac30"
server_key_iv: "04d18ec6973cff66237f58ef764a3074"
server_key_version: "encryptionkey@0"
I20260806 22:47:17.894189 15567 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:47:17.895061 15590 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:17.895390 15567 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:47:17.895510 15567 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/master-1/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/master-1/wal
uuid: "0016a76e8ad5476995c85a46a1fdcb6b"
format_stamp: "Formatted at 2026-08-06 22:47:17 on dist-test-slave-zh2d"
server_key: "4d0c5499ba484d8a1c4e927f4686ac30"
server_key_iv: "04d18ec6973cff66237f58ef764a3074"
server_key_version: "encryptionkey@0"
I20260806 22:47:17.895597 15567 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:47:17.916618 15567 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:47:17.918979 15567 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:47:17.919160 15567 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:47:17.923647 15567 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.125:42247
I20260806 22:47:17.923758 15656 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.125:42247 every 8 connection(s)
I20260806 22:47:17.924100 15567 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/master-1/data/info.pb
I20260806 22:47:17.925288 15657 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:47:17.927065 15657 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.40.126" port: 38077 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:47:17.927789 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 15567
Aug 06 22:47:17 dist-test-slave-zh2d krb5kdc[14880](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056437, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.40.125@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM
Aug 06 22:47:17 dist-test-slave-zh2d krb5kdc[14880](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056437, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.6.40.125@KRBTEST.COM
I20260806 22:47:17.937937 14994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:17.927996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48066 (local address 127.6.40.126:38077)
0806 22:47:17.928102 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:17.928105 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:17.928391 (+ 286us) server_negotiation.cc:407] Connection header received
0806 22:47:17.929214 (+ 823us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:17.929216 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:17.929232 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:17.929301 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:17.929970 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:17.930463 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:17.931610 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:17.931891 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:17.931946 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:17.934607 (+ 2661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:17.934609 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:17.934609 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:17.934617 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:17.936307 (+ 1690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:17.936805 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:17.936806 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:17.936807 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:17.936920 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:17.937186 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:17.937188 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:17.937188 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:17.937362 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:17.937453 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:17.937739 (+ 286us) server_negotiation.cc:299] Negotiation successful
0806 22:47:17.937824 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 22:47:17.938159 15660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:17.927955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:38077 (local address 127.0.0.1:48066)
0806 22:47:17.928191 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:17.928245 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:17.929087 (+ 842us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:17.929347 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:17.929350 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:17.929655 (+ 305us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:17.929858 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:17.929865 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:17.930568 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:17.930570 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:17.931521 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:17.931524 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:17.931992 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:17.931993 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:17.932137 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:17.932781 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:17.932791 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:17.934510 (+ 1719us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:47:17.936420 (+ 1910us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:17.936427 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:17.936429 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:17.936726 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:17.936990 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:17.936991 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:17.936991 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:17.937112 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:17.937443 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:47:17.937451 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:47:17.937602 (+ 151us) client_negotiation.cc:769] Sending connection context
0806 22:47:17.937716 (+ 114us) client_negotiation.cc:240] Negotiation successful
0806 22:47:17.937886 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":103,"threads_started":1}
I20260806 22:47:17.938194 15662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:17.928487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52550 (local address 127.6.40.125:42247)
0806 22:47:17.928669 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:17.928674 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:17.928723 (+ 49us) server_negotiation.cc:407] Connection header received
0806 22:47:17.929090 (+ 367us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:17.929099 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:17.929132 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:17.929232 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:17.929869 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:17.930990 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:17.931789 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:17.932244 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:17.932329 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:17.935073 (+ 2744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:17.935084 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:17.935085 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:17.935097 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:17.936786 (+ 1689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:17.937242 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:17.937245 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:17.937246 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:17.937312 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:17.937546 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:17.937547 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:17.937548 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:17.937704 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:17.937876 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:17.937975 (+ 99us) server_negotiation.cc:299] Negotiation successful
0806 22:47:17.938064 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":114,"threads_started":1}
I20260806 22:47:17.939034 15657 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.40.125" port: 42247 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 22:47:17 dist-test-slave-zh2d krb5kdc[14880](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056437, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.40.125@KRBTEST.COM for oryx/127.6.40.125@KRBTEST.COM
I20260806 22:47:17.945734 15660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:17.939317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.125:42247 (local address 127.0.0.1:52558)
0806 22:47:17.939377 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:17.939391 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:17.939434 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:17.939572 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:17.939572 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:17.939697 (+ 125us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:17.939749 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:17.939750 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:17.940345 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:17.940346 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:17.940967 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:17.940969 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:17.941319 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:17.941320 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:17.941399 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:17.941774 (+ 375us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:17.941778 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:17.943529 (+ 1751us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:47:17.944820 (+ 1291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:17.944821 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:17.944821 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:17.945053 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:17.945231 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:17.945232 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:17.945232 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:17.945285 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:17.945500 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:47:17.945501 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:47:17.945536 (+ 35us) client_negotiation.cc:769] Sending connection context
0806 22:47:17.945588 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 22:47:17.945666 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260806 22:47:17.945837 15662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:17.939372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52558 (local address 127.6.40.125:42247)
0806 22:47:17.939407 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:17.939409 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:17.939421 (+ 12us) server_negotiation.cc:407] Connection header received
0806 22:47:17.939511 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:17.939512 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:17.939526 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:17.939561 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:17.939804 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:17.940274 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:17.941044 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:17.941236 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:17.941289 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:17.943660 (+ 2371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:17.943661 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:17.943661 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:17.943666 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:17.944747 (+ 1081us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:17.945121 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:17.945122 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:17.945122 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:17.945173 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:17.945323 (+ 150us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:17.945323 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:17.945323 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:17.945447 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:17.945498 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:17.945602 (+ 104us) server_negotiation.cc:299] Negotiation successful
0806 22:47:17.945712 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":48,"server-negotiator.queue_time_us":17}
I20260806 22:47:17.947190 15657 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0016a76e8ad5476995c85a46a1fdcb6b: Bootstrap starting.
I20260806 22:47:17.948076 15657 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0016a76e8ad5476995c85a46a1fdcb6b: Neither blocks nor log segments found. Creating new log.
I20260806 22:47:17.948369 15657 log.cc:824] T 00000000000000000000000000000000 P 0016a76e8ad5476995c85a46a1fdcb6b: Log is configured to *not* fsync() on all Append() calls
I20260806 22:47:17.949190 15657 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0016a76e8ad5476995c85a46a1fdcb6b: No bootstrap required, opened a new log
I20260806 22:47:17.950676 15657 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0016a76e8ad5476995c85a46a1fdcb6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb7a8590eeb54daab448b3b04d3dfacf" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 38077 } } peers { permanent_uuid: "0016a76e8ad5476995c85a46a1fdcb6b" member_type: VOTER last_known_addr { host: "127.6.40.125" port: 42247 } }
I20260806 22:47:17.950800 15657 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0016a76e8ad5476995c85a46a1fdcb6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:47:17.950824 15657 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0016a76e8ad5476995c85a46a1fdcb6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0016a76e8ad5476995c85a46a1fdcb6b, State: Initialized, Role: FOLLOWER
I20260806 22:47:17.950958 15657 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0016a76e8ad5476995c85a46a1fdcb6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb7a8590eeb54daab448b3b04d3dfacf" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 38077 } } peers { permanent_uuid: "0016a76e8ad5476995c85a46a1fdcb6b" member_type: VOTER last_known_addr { host: "127.6.40.125" port: 42247 } }
I20260806 22:47:17.951292 15657 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0016a76e8ad5476995c85a46a1fdcb6b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:47:17.951270 15667 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0016a76e8ad5476995c85a46a1fdcb6b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb7a8590eeb54daab448b3b04d3dfacf" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 38077 } } peers { permanent_uuid: "0016a76e8ad5476995c85a46a1fdcb6b" member_type: VOTER last_known_addr { host: "127.6.40.125" port: 42247 } } }
I20260806 22:47:17.951395 15667 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0016a76e8ad5476995c85a46a1fdcb6b [sys.catalog]: This master's current role is: FOLLOWER
W20260806 22:47:17.953126 15686 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0016a76e8ad5476995c85a46a1fdcb6b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:47:17.953229 15686 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:47:17.954996 15567 master_runner.cc:428] Detected that this master 0016a76e8ad5476995c85a46a1fdcb6b is joining an existing cluster
I20260806 22:47:17.955086 15567 master_runner.cc:432] Initiating AddMaster RPC to add 127.6.40.125:42247
I20260806 22:47:17.963858 14994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:17.956165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48082 (local address 127.6.40.126:38077)
0806 22:47:17.956212 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:17.956214 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:17.956352 (+ 138us) server_negotiation.cc:407] Connection header received
0806 22:47:17.956474 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:17.956475 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:17.956485 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:17.956513 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:17.957161 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:17.957651 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:17.958758 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:17.959095 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:17.959130 (+ 35us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:17.960680 (+ 1550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:17.960682 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:17.960682 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:17.960689 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:17.962452 (+ 1763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:17.962897 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:17.962906 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:17.962906 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:17.962984 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:17.963203 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:17.963204 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:17.963205 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:17.963372 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:17.963453 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:17.963662 (+ 209us) server_negotiation.cc:299] Negotiation successful
0806 22:47:17.963751 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 22:47:17.964078 15693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:17.956136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:38077 (local address 127.0.0.1:48082)
0806 22:47:17.956281 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:17.956292 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:17.956382 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:17.956601 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:17.956602 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:17.956951 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:17.957057 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:17.957059 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:17.957737 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:17.957738 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:17.958659 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:17.958661 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:17.959237 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:17.959238 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:17.959315 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:17.959962 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:17.959967 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:17.960581 (+ 614us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:47:17.962558 (+ 1977us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:17.962559 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:17.962560 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:17.962822 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:17.963062 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:17.963063 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:17.963063 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:17.963119 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:17.963489 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:47:17.963491 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:47:17.963544 (+ 53us) client_negotiation.cc:769] Sending connection context
0806 22:47:17.963628 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 22:47:17.963739 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"mutex_wait_us":3,"thread_start_us":89,"threads_started":1}
I20260806 22:47:17.964143 15662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:17.956411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52572 (local address 127.6.40.125:42247)
0806 22:47:17.956455 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:17.956457 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:17.956466 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:47:17.956491 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:17.956492 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:17.956505 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:17.956571 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:17.957144 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:17.957946 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:17.958887 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:17.959221 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:17.959300 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:17.960655 (+ 1355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:17.960657 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:17.960657 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:17.960665 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:17.962921 (+ 2256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:17.963287 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:17.963288 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:17.963288 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:17.963335 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:17.963558 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:17.963559 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:17.963560 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:17.963679 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:17.963742 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:17.963914 (+ 172us) server_negotiation.cc:299] Negotiation successful
0806 22:47:17.963997 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 22:47:17.964556 15695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:17.956200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.125:42247 (local address 127.0.0.1:52572)
0806 22:47:17.956355 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:17.956365 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:17.956405 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:17.956588 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:17.956589 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:17.956969 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:17.957057 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:17.957059 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:17.958046 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:17.958047 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:17.958801 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:17.958803 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:17.959307 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:17.959307 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:17.959362 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:17.960062 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:17.960067 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:17.960553 (+ 486us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:47:17.963009 (+ 2456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:17.963010 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:17.963011 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:17.963221 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:17.963416 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:17.963417 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:17.963418 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:17.963493 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:17.963763 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:47:17.963764 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:47:17.963797 (+ 33us) client_negotiation.cc:769] Sending connection context
0806 22:47:17.964098 (+ 301us) client_negotiation.cc:240] Negotiation successful
0806 22:47:17.964226 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":119,"threads_started":1}
Aug 06 22:47:17 dist-test-slave-zh2d krb5kdc[14880](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056437, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.40.126@KRBTEST.COM for oryx/127.6.40.125@KRBTEST.COM
I20260806 22:47:17.974421 15662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:17.967135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52582 (local address 127.6.40.125:42247)
0806 22:47:17.967177 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:17.967179 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:17.967308 (+ 129us) server_negotiation.cc:407] Connection header received
0806 22:47:17.967401 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:17.967402 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:17.967414 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:17.967470 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:17.967896 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:17.968348 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:17.969230 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:17.969420 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:17.969495 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:17.971557 (+ 2062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:17.971559 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:17.971560 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:17.971566 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:17.973030 (+ 1464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:17.973503 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:17.973504 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:17.973505 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:17.973569 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:17.973845 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:17.973846 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:17.973846 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:17.973976 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:17.974049 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:17.974243 (+ 194us) server_negotiation.cc:299] Negotiation successful
0806 22:47:17.974323 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 22:47:17.974500 15696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:17.967043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.125:42247 (local address 127.0.0.1:52582)
0806 22:47:17.967190 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:17.967233 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:17.967334 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:17.967478 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:17.967482 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:17.967761 (+ 279us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:17.967813 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:17.967814 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:17.968433 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:17.968434 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:17.969131 (+ 697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:17.969133 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:17.969507 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:17.969508 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:17.969593 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:17.970083 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:17.970087 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:17.971464 (+ 1377us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:47:17.973139 (+ 1675us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:17.973147 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:17.973149 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:17.973420 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:17.973639 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:17.973640 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:17.973640 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:17.973764 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:17.974055 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:47:17.974064 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:47:17.974119 (+ 55us) client_negotiation.cc:769] Sending connection context
0806 22:47:17.974217 (+ 98us) client_negotiation.cc:240] Negotiation successful
0806 22:47:17.974339 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":78,"threads_started":1}
I20260806 22:47:17.975099 14938 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.6.40.125:42247: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "0016a76e8ad5476995c85a46a1fdcb6b"
member_type: NON_VOTER
last_known_addr {
host: "127.6.40.125"
port: 42247
}
attrs {
promote: true
}
}
dest_uuid: "eb7a8590eeb54daab448b3b04d3dfacf"
cas_config_opid_index: -1
I20260806 22:47:17.975422 14938 consensus_queue.cc:237] T 00000000000000000000000000000000 P eb7a8590eeb54daab448b3b04d3dfacf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "eb7a8590eeb54daab448b3b04d3dfacf" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 38077 } } peers { permanent_uuid: "0016a76e8ad5476995c85a46a1fdcb6b" member_type: NON_VOTER last_known_addr { host: "127.6.40.125" port: 42247 } attrs { promote: true } }
I20260806 22:47:17.975888 15698 raft_consensus.cc:2932] T 00000000000000000000000000000000 P eb7a8590eeb54daab448b3b04d3dfacf [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 0016a76e8ad5476995c85a46a1fdcb6b (127.6.40.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "eb7a8590eeb54daab448b3b04d3dfacf" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 38077 } } peers { permanent_uuid: "0016a76e8ad5476995c85a46a1fdcb6b" member_type: NON_VOTER last_known_addr { host: "127.6.40.125" port: 42247 } attrs { promote: true } } }
I20260806 22:47:17.976293 15624 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0016a76e8ad5476995c85a46a1fdcb6b [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:47:17.977135 15698 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.6.40.125:42247
I20260806 22:47:17.977268 15697 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb7a8590eeb54daab448b3b04d3dfacf [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "eb7a8590eeb54daab448b3b04d3dfacf" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "eb7a8590eeb54daab448b3b04d3dfacf" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 38077 } } peers { permanent_uuid: "0016a76e8ad5476995c85a46a1fdcb6b" member_type: NON_VOTER last_known_addr { host: "127.6.40.125" port: 42247 } attrs { promote: true } } }
I20260806 22:47:17.977370 15697 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb7a8590eeb54daab448b3b04d3dfacf [sys.catalog]: This master's current role is: LEADER
I20260806 22:47:17.977447 15624 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0016a76e8ad5476995c85a46a1fdcb6b [term 1 FOLLOWER]: Refusing update from remote peer eb7a8590eeb54daab448b3b04d3dfacf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260806 22:47:17.977780 15567 master.cc:537] Master@127.6.40.125:42247 shutting down...
I20260806 22:47:17.977869 15697 consensus_queue.cc:1035] T 00000000000000000000000000000000 P eb7a8590eeb54daab448b3b04d3dfacf [LEADER]: Connected to new peer: Peer: permanent_uuid: "0016a76e8ad5476995c85a46a1fdcb6b" member_type: NON_VOTER last_known_addr { host: "127.6.40.125" port: 42247 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 22:47:17.979588 14921 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260806 22:47:17.980142 14921 consensus_peers.cc:487] T 00000000000000000000000000000000 P eb7a8590eeb54daab448b3b04d3dfacf -> Peer 0016a76e8ad5476995c85a46a1fdcb6b (127.6.40.125:42247): Couldn't send request to peer 0016a76e8ad5476995c85a46a1fdcb6b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260806 22:47:17.980635 15567 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 0016a76e8ad5476995c85a46a1fdcb6b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:47:17.980727 15567 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 0016a76e8ad5476995c85a46a1fdcb6b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:47:17.980791 15567 tablet_replica.cc:330] T 00000000000000000000000000000000 P 0016a76e8ad5476995c85a46a1fdcb6b: stopping tablet replica
I20260806 22:47:17.993700 15567 master.cc:552] Master@127.6.40.125:42247 shutdown complete.
I20260806 22:47:17.993762 15567 master_runner.cc:305] Clearing existing system tablet
I20260806 22:47:17.993912 15567 fs_manager.cc:359] Metadata directory not provided
I20260806 22:47:17.993978 15567 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:47:17.994853 15567 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:47:17.995469 15702 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:17.995633 15567 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:47:17.995704 15567 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:47:17.995733 15567 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/master-1/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/master-1/wal
uuid: "0016a76e8ad5476995c85a46a1fdcb6b"
format_stamp: "Formatted at 2026-08-06 22:47:17 on dist-test-slave-zh2d"
server_key: "4d0c5499ba484d8a1c4e927f4686ac30"
server_key_iv: "04d18ec6973cff66237f58ef764a3074"
server_key_version: "encryptionkey@0"
I20260806 22:47:17.996661 15567 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 0016a76e8ad5476995c85a46a1fdcb6b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:47:17.998905 15567 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 0016a76e8ad5476995c85a46a1fdcb6b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 22:47:17.999001 15567 log.cc:1196] T 00000000000000000000000000000000 P 0016a76e8ad5476995c85a46a1fdcb6b: Deleting WAL directory at /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260806 22:47:17.999225 15567 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 0016a76e8ad5476995c85a46a1fdcb6b: Deleting consensus metadata
I20260806 22:47:17.999366 15567 master_runner.cc:315] Copying system tablet from 127.6.40.126:38077
I20260806 22:47:18.000078 15567 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 0016a76e8ad5476995c85a46a1fdcb6b: tablet copy: Beginning tablet copy session from remote peer at address 127.6.40.126:38077
I20260806 22:47:18.007479 15708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:18.000518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:38077 (local address 127.0.0.1:48092)
0806 22:47:18.000659 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:18.000670 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:18.000741 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:18.000937 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:18.000938 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:18.001129 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:18.001193 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:18.001196 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.002316 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.002317 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:18.003042 (+ 725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:18.003045 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.003578 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.003579 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:18.003638 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:18.004184 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:18.004188 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:18.004518 (+ 330us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:47:18.006444 (+ 1926us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:18.006445 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:18.006445 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:18.006628 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:18.006846 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:18.006847 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:18.006847 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:18.006914 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:18.007207 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:47:18.007207 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:47:18.007249 (+ 42us) client_negotiation.cc:769] Sending connection context
0806 22:47:18.007301 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 22:47:18.007388 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":97,"threads_started":1}
I20260806 22:47:18.007551 14994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:18.000573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48092 (local address 127.6.40.126:38077)
0806 22:47:18.000636 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:18.000638 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:18.000750 (+ 112us) server_negotiation.cc:407] Connection header received
0806 22:47:18.000785 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:18.000786 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:18.000799 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:18.000930 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:18.001278 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.002209 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.003127 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.003482 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.003561 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:18.004614 (+ 1053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:18.004616 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:18.004617 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:18.004626 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:18.006346 (+ 1720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:18.006700 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:18.006702 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:18.006702 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:18.006772 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:18.006990 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:18.006991 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:18.006991 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:18.007139 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:18.007216 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:18.007349 (+ 133us) server_negotiation.cc:299] Negotiation successful
0806 22:47:18.007447 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 22:47:18.008231 14968 tablet_copy_service.cc:140] P eb7a8590eeb54daab448b3b04d3dfacf: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 0016a76e8ad5476995c85a46a1fdcb6b ({username='oryx', principal='oryx/127.6.40.125@KRBTEST.COM'} at 127.0.0.1:48092)
I20260806 22:47:18.008383 14968 tablet_copy_service.cc:161] P eb7a8590eeb54daab448b3b04d3dfacf: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 0016a76e8ad5476995c85a46a1fdcb6b at {username='oryx', principal='oryx/127.6.40.125@KRBTEST.COM'} at 127.0.0.1:48092: session id = 0016a76e8ad5476995c85a46a1fdcb6b-00000000000000000000000000000000
I20260806 22:47:18.009328 14968 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P eb7a8590eeb54daab448b3b04d3dfacf: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 22:47:18.010068 15567 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 22:47:18.011615 15567 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 0016a76e8ad5476995c85a46a1fdcb6b: tablet copy: Starting download of 0 data blocks...
I20260806 22:47:18.011794 15567 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 0016a76e8ad5476995c85a46a1fdcb6b: tablet copy: Starting download of 1 WAL segments...
I20260806 22:47:18.012908 15567 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 0016a76e8ad5476995c85a46a1fdcb6b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 22:47:18.013893 14968 tablet_copy_service.cc:342] P eb7a8590eeb54daab448b3b04d3dfacf: Request end of tablet copy session 0016a76e8ad5476995c85a46a1fdcb6b-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.6.40.125@KRBTEST.COM'} at 127.0.0.1:48092
I20260806 22:47:18.013968 14968 tablet_copy_service.cc:433] P eb7a8590eeb54daab448b3b04d3dfacf: ending tablet copy session 0016a76e8ad5476995c85a46a1fdcb6b-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 0016a76e8ad5476995c85a46a1fdcb6b
I20260806 22:47:18.017175 15567 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:47:18.018425 15711 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:18.018535 15712 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:18.018716 15567 server_base.cc:1034] running on GCE node
W20260806 22:47:18.020983 15714 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:18.021160 15567 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:47:18.021430 15567 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:47:18.022598 15567 hybrid_clock.cc:648] HybridClock initialized: now 1786056438022587 us; error 32 us; skew 500 ppm
Aug 06 22:47:18 dist-test-slave-zh2d krb5kdc[14880](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056438, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.40.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:47:18.024184 15567 init.cc:377] Logged in from keytab as oryx/127.6.40.125@KRBTEST.COM (short username oryx)
I20260806 22:47:18.024537 15567 webserver.cc:460] Webserver started at http://127.6.40.125:33349/ using document root <none> and password file <none>
I20260806 22:47:18.024693 15567 fs_manager.cc:359] Metadata directory not provided
I20260806 22:47:18.024739 15567 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:47:18.025483 15567 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:18.026109 15723 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:18.026283 15567 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:47:18.026343 15567 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/master-1/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/master-1/wal
uuid: "0016a76e8ad5476995c85a46a1fdcb6b"
format_stamp: "Formatted at 2026-08-06 22:47:17 on dist-test-slave-zh2d"
server_key: "4d0c5499ba484d8a1c4e927f4686ac30"
server_key_iv: "04d18ec6973cff66237f58ef764a3074"
server_key_version: "encryptionkey@0"
I20260806 22:47:18.026437 15567 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:47:18.030715 15567 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:47:18.032637 15567 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:47:18.036500 15567 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.125:42247
I20260806 22:47:18.036931 15790 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.125:42247 every 8 connection(s)
I20260806 22:47:18.037046 15567 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/master-1/data/info.pb
I20260806 22:47:18.037807 15793 sys_catalog.cc:263] Verifying existing consensus state
I20260806 22:47:18.038242 15793 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0016a76e8ad5476995c85a46a1fdcb6b: Bootstrap starting.
I20260806 22:47:18.041033 15793 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0016a76e8ad5476995c85a46a1fdcb6b: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 22:47:18.041285 15793 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0016a76e8ad5476995c85a46a1fdcb6b: Bootstrap complete.
I20260806 22:47:18.041603 15793 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0016a76e8ad5476995c85a46a1fdcb6b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "eb7a8590eeb54daab448b3b04d3dfacf" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 38077 } } peers { permanent_uuid: "0016a76e8ad5476995c85a46a1fdcb6b" member_type: NON_VOTER last_known_addr { host: "127.6.40.125" port: 42247 } attrs { promote: true } }
I20260806 22:47:18.041703 15793 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0016a76e8ad5476995c85a46a1fdcb6b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 0016a76e8ad5476995c85a46a1fdcb6b, State: Initialized, Role: LEARNER
I20260806 22:47:18.041742 15793 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0016a76e8ad5476995c85a46a1fdcb6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "eb7a8590eeb54daab448b3b04d3dfacf" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 38077 } } peers { permanent_uuid: "0016a76e8ad5476995c85a46a1fdcb6b" member_type: NON_VOTER last_known_addr { host: "127.6.40.125" port: 42247 } attrs { promote: true } }
I20260806 22:47:18.041970 15793 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0016a76e8ad5476995c85a46a1fdcb6b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:47:18.041966 15799 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0016a76e8ad5476995c85a46a1fdcb6b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "eb7a8590eeb54daab448b3b04d3dfacf" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 38077 } } peers { permanent_uuid: "0016a76e8ad5476995c85a46a1fdcb6b" member_type: NON_VOTER last_known_addr { host: "127.6.40.125" port: 42247 } attrs { promote: true } } }
I20260806 22:47:18.042212 15799 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0016a76e8ad5476995c85a46a1fdcb6b [sys.catalog]: This master's current role is: LEARNER
I20260806 22:47:18.043670 15816 catalog_manager.cc:1259] Loaded cluster ID: 16746f209dcb48eb906c4d27294225e8
I20260806 22:47:18.043718 15816 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0016a76e8ad5476995c85a46a1fdcb6b: loading cluster ID for follower catalog manager: success
I20260806 22:47:18.044730 15816 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0016a76e8ad5476995c85a46a1fdcb6b: acquiring CA information for follower catalog manager: success
I20260806 22:47:18.045092 15816 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0016a76e8ad5476995c85a46a1fdcb6b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 22:47:18.049389 15798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:18.040427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52606 (local address 127.6.40.125:42247)
0806 22:47:18.040566 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:18.040568 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:18.040580 (+ 12us) server_negotiation.cc:407] Connection header received
0806 22:47:18.040608 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:18.040610 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:18.040621 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:18.040718 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:18.041533 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.042662 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.043967 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.044542 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.044621 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:18.046552 (+ 1931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:18.046553 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:18.046553 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:18.046559 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:18.048098 (+ 1539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:18.048536 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:18.048537 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:18.048537 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:18.048593 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:18.048892 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:18.048892 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:18.048893 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:18.049006 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:18.049086 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:18.049252 (+ 166us) server_negotiation.cc:299] Negotiation successful
0806 22:47:18.049325 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":261,"server-negotiator.queue_time_us":110,"thread_start_us":91,"threads_started":1}
WARNING: no policy specified for oryx/127.6.40.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.40.124@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.40.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/krb5kdc/oryx_127.6.40.124.keytab.
Entry for principal oryx/127.6.40.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/krb5kdc/oryx_127.6.40.124.keytab.
I20260806 22:47:18.080742 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.124:42743
--webserver_interface=127.6.40.124
--webserver_port=0
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/krb5kdc/oryx_127.6.40.124.keytab
--principal=oryx/127.6.40.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:34373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.40.126:38077,127.6.40.125:42247,127.6.40.124:42743
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:47:18.177340 15819 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:47:18.177572 15819 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:47:18.177597 15819 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:47:18.178807 15819 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:47:18.178853 15819 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:47:18.178869 15819 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:47:18.178884 15819 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:47:18.178896 15819 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:47:18.180506 15819 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:34373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.6.40.126:38077,127.6.40.125:42247,127.6.40.124:42743
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/krb5kdc/oryx_127.6.40.124.keytab
--principal=oryx/127.6.40.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.124:42743
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.6.40.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15819
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:47:18.180713 15819 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:47:18.180929 15819 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:47:18.186358 15827 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:18.186434 15830 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:18.186486 15828 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:18.186578 15819 server_base.cc:1034] running on GCE node
I20260806 22:47:18.186947 15819 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:47:18.187283 15819 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:47:18.188474 15819 hybrid_clock.cc:648] HybridClock initialized: now 1786056438188459 us; error 44 us; skew 500 ppm
Aug 06 22:47:18 dist-test-slave-zh2d krb5kdc[14880](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056438, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.40.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:47:18.191212 15819 init.cc:377] Logged in from keytab as oryx/127.6.40.124@KRBTEST.COM (short username oryx)
I20260806 22:47:18.191642 15819 webserver.cc:460] Webserver started at http://127.6.40.124:33197/ using document root <none> and password file <none>
I20260806 22:47:18.191884 15819 fs_manager.cc:359] Metadata directory not provided
I20260806 22:47:18.191931 15819 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:47:18.192111 15819 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:47:18.193265 15819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/master-2/data/instance:
uuid: "cca9c82ca527425abfcddb8d93dd4b64"
format_stamp: "Formatted at 2026-08-06 22:47:18 on dist-test-slave-zh2d"
server_key: "4b9451f51f12cf73d480e409e78a1a96"
server_key_iv: "6331bdf9c05e17707277d1bc8400c999"
server_key_version: "encryptionkey@0"
I20260806 22:47:18.193634 15819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/master-2/wal/instance:
uuid: "cca9c82ca527425abfcddb8d93dd4b64"
format_stamp: "Formatted at 2026-08-06 22:47:18 on dist-test-slave-zh2d"
server_key: "4b9451f51f12cf73d480e409e78a1a96"
server_key_iv: "6331bdf9c05e17707277d1bc8400c999"
server_key_version: "encryptionkey@0"
I20260806 22:47:18.195127 15819 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:47:18.195930 15839 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:18.196136 15819 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:47:18.196229 15819 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/master-2/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/master-2/wal
uuid: "cca9c82ca527425abfcddb8d93dd4b64"
format_stamp: "Formatted at 2026-08-06 22:47:18 on dist-test-slave-zh2d"
server_key: "4b9451f51f12cf73d480e409e78a1a96"
server_key_iv: "6331bdf9c05e17707277d1bc8400c999"
server_key_version: "encryptionkey@0"
I20260806 22:47:18.196313 15819 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:47:18.235354 15819 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:47:18.237442 15819 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:47:18.237608 15819 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:47:18.244206 15819 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.124:42743
I20260806 22:47:18.244236 15910 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.124:42743 every 8 connection(s)
I20260806 22:47:18.244683 15819 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/master-2/data/info.pb
I20260806 22:47:18.245836 15911 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:47:18.247634 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 15819
I20260806 22:47:18.247890 15911 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.40.126" port: 38077 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 22:47:18 dist-test-slave-zh2d krb5kdc[14880](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056438, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.40.124@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM
Aug 06 22:47:18 dist-test-slave-zh2d krb5kdc[14880](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056437, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.6.40.124@KRBTEST.COM
I20260806 22:47:18.259368 14994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:18.249230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48094 (local address 127.6.40.126:38077)
0806 22:47:18.249288 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:18.249290 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:18.249460 (+ 170us) server_negotiation.cc:407] Connection header received
0806 22:47:18.249903 (+ 443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:18.249905 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:18.249919 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:18.249995 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:18.250606 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.251093 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.252422 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.252735 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.252806 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:18.255345 (+ 2539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:18.255346 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:18.255346 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:18.255372 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:18.257358 (+ 1986us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:18.258059 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:18.258061 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:18.258061 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:18.258157 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:18.258513 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:18.258514 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:18.258515 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:18.258711 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:18.258778 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:18.259145 (+ 367us) server_negotiation.cc:299] Negotiation successful
0806 22:47:18.259235 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 22:47:18.259550 15915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:18.249089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:38077 (local address 127.0.0.1:48094)
0806 22:47:18.249314 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:18.249361 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:18.249736 (+ 375us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:18.250035 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:18.250037 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:18.250300 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:18.250478 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:18.250481 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.251193 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.251194 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:18.252341 (+ 1147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:18.252343 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.252819 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.252820 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:18.252993 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:18.253640 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:18.253648 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:18.255259 (+ 1611us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:47:18.257449 (+ 2190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:18.257458 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:18.257460 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:18.257894 (+ 434us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:18.258261 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:18.258262 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:18.258262 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:18.258432 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:18.258812 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:47:18.258820 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:47:18.258979 (+ 159us) client_negotiation.cc:769] Sending connection context
0806 22:47:18.259098 (+ 119us) client_negotiation.cc:240] Negotiation successful
0806 22:47:18.259272 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"mutex_wait_us":143,"thread_start_us":141,"threads_started":1}
I20260806 22:47:18.260344 15911 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.40.125" port: 42247 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:47:18.260444 15914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:18.248545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56702 (local address 127.6.40.124:42743)
0806 22:47:18.248963 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:18.248972 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:18.249005 (+ 33us) server_negotiation.cc:407] Connection header received
0806 22:47:18.249734 (+ 729us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:18.249744 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:18.249770 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:18.249862 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:18.250554 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.251932 (+ 1378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.252754 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.253308 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.253364 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:18.255937 (+ 2573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:18.255947 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:18.255947 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:18.255963 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:18.258783 (+ 2820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:18.259213 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:18.259216 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:18.259217 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:18.259279 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:18.259497 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:18.259498 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:18.259498 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:18.259661 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:18.260179 (+ 518us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:18.260219 (+ 40us) server_negotiation.cc:299] Negotiation successful
0806 22:47:18.260307 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":311,"threads_started":1}
Aug 06 22:47:18 dist-test-slave-zh2d krb5kdc[14880](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056438, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.40.124@KRBTEST.COM for oryx/127.6.40.125@KRBTEST.COM
I20260806 22:47:18.266880 15915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:18.260738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.125:42247 (local address 127.0.0.1:52610)
0806 22:47:18.260781 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:18.260790 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:18.260821 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:18.261045 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:18.261046 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:18.261204 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:18.261244 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:18.261246 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.261856 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.261856 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:18.262335 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:18.262336 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.262687 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.262688 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:18.262740 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:18.263148 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:18.263152 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:18.264385 (+ 1233us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:47:18.265944 (+ 1559us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:18.265945 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:18.265945 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:18.266109 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:18.266281 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:18.266282 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:18.266282 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:18.266329 (+ 47us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:18.266607 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:47:18.266608 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:47:18.266642 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 22:47:18.266693 (+ 51us) client_negotiation.cc:240] Negotiation successful
0806 22:47:18.266761 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260806 22:47:18.266881 15798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:18.260825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52610 (local address 127.6.40.125:42247)
0806 22:47:18.260892 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:18.260894 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:18.260912 (+ 18us) server_negotiation.cc:407] Connection header received
0806 22:47:18.260940 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:18.260941 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:18.260952 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:18.261018 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:18.261317 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.261793 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.262414 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.262617 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.262673 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:18.264472 (+ 1799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:18.264473 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:18.264474 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:18.264481 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:18.265855 (+ 1374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:18.266171 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:18.266172 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:18.266172 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:18.266224 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:18.266383 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:18.266383 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:18.266383 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:18.266535 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:18.266619 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:18.266714 (+ 95us) server_negotiation.cc:299] Negotiation successful
0806 22:47:18.266783 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 22:47:18.267395 15911 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.40.124" port: 42743 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 22:47:18 dist-test-slave-zh2d krb5kdc[14880](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056438, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.40.124@KRBTEST.COM for oryx/127.6.40.124@KRBTEST.COM
I20260806 22:47:18.273308 15915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:18.267676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.124:42743 (local address 127.0.0.1:56710)
0806 22:47:18.267725 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:18.267736 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:18.267781 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:18.267888 (+ 107us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:18.267888 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:18.268000 (+ 112us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:18.268028 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:18.268029 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.268670 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.268671 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:18.269180 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:18.269182 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.269627 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.269628 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:18.269664 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:18.270038 (+ 374us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:18.270047 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:18.271100 (+ 1053us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:47:18.272307 (+ 1207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:18.272308 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:18.272308 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:18.272467 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:18.272712 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:18.272712 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:18.272713 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:18.272761 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:18.273095 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:47:18.273096 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:47:18.273129 (+ 33us) client_negotiation.cc:769] Sending connection context
0806 22:47:18.273179 (+ 50us) client_negotiation.cc:240] Negotiation successful
0806 22:47:18.273251 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260806 22:47:18.273505 15914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:18.267704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56710 (local address 127.6.40.124:42743)
0806 22:47:18.267753 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:18.267754 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:18.267766 (+ 12us) server_negotiation.cc:407] Connection header received
0806 22:47:18.267825 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:18.267825 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:18.267832 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:18.267890 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:18.268082 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.268606 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.269247 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.269560 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.269618 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:18.271175 (+ 1557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:18.271176 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:18.271176 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:18.271182 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:18.272240 (+ 1058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:18.272559 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:18.272561 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:18.272561 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:18.272634 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:18.272824 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:18.272825 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:18.272825 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:18.273012 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:18.273099 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:18.273211 (+ 112us) server_negotiation.cc:299] Negotiation successful
0806 22:47:18.273351 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":26,"server-negotiator.queue_time_us":30}
I20260806 22:47:18.274971 15911 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cca9c82ca527425abfcddb8d93dd4b64: Bootstrap starting.
I20260806 22:47:18.275867 15911 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cca9c82ca527425abfcddb8d93dd4b64: Neither blocks nor log segments found. Creating new log.
I20260806 22:47:18.276288 15911 log.cc:824] T 00000000000000000000000000000000 P cca9c82ca527425abfcddb8d93dd4b64: Log is configured to *not* fsync() on all Append() calls
I20260806 22:47:18.277345 15911 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cca9c82ca527425abfcddb8d93dd4b64: No bootstrap required, opened a new log
I20260806 22:47:18.278865 15911 raft_consensus.cc:348] T 00000000000000000000000000000000 P cca9c82ca527425abfcddb8d93dd4b64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb7a8590eeb54daab448b3b04d3dfacf" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 38077 } } peers { permanent_uuid: "0016a76e8ad5476995c85a46a1fdcb6b" member_type: VOTER last_known_addr { host: "127.6.40.125" port: 42247 } } peers { permanent_uuid: "cca9c82ca527425abfcddb8d93dd4b64" member_type: VOTER last_known_addr { host: "127.6.40.124" port: 42743 } }
I20260806 22:47:18.279036 15911 raft_consensus.cc:374] T 00000000000000000000000000000000 P cca9c82ca527425abfcddb8d93dd4b64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:47:18.279100 15911 raft_consensus.cc:729] T 00000000000000000000000000000000 P cca9c82ca527425abfcddb8d93dd4b64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cca9c82ca527425abfcddb8d93dd4b64, State: Initialized, Role: FOLLOWER
I20260806 22:47:18.279253 15911 consensus_queue.cc:260] T 00000000000000000000000000000000 P cca9c82ca527425abfcddb8d93dd4b64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb7a8590eeb54daab448b3b04d3dfacf" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 38077 } } peers { permanent_uuid: "0016a76e8ad5476995c85a46a1fdcb6b" member_type: VOTER last_known_addr { host: "127.6.40.125" port: 42247 } } peers { permanent_uuid: "cca9c82ca527425abfcddb8d93dd4b64" member_type: VOTER last_known_addr { host: "127.6.40.124" port: 42743 } }
I20260806 22:47:18.279624 15911 sys_catalog.cc:564] T 00000000000000000000000000000000 P cca9c82ca527425abfcddb8d93dd4b64 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:47:18.279621 15917 sys_catalog.cc:455] T 00000000000000000000000000000000 P cca9c82ca527425abfcddb8d93dd4b64 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb7a8590eeb54daab448b3b04d3dfacf" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 38077 } } peers { permanent_uuid: "0016a76e8ad5476995c85a46a1fdcb6b" member_type: VOTER last_known_addr { host: "127.6.40.125" port: 42247 } } peers { permanent_uuid: "cca9c82ca527425abfcddb8d93dd4b64" member_type: VOTER last_known_addr { host: "127.6.40.124" port: 42743 } } }
I20260806 22:47:18.279794 15917 sys_catalog.cc:458] T 00000000000000000000000000000000 P cca9c82ca527425abfcddb8d93dd4b64 [sys.catalog]: This master's current role is: FOLLOWER
W20260806 22:47:18.281611 15934 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cca9c82ca527425abfcddb8d93dd4b64: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:47:18.281677 15934 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:47:18.284150 15819 master_runner.cc:428] Detected that this master cca9c82ca527425abfcddb8d93dd4b64 is joining an existing cluster
I20260806 22:47:18.284204 15819 master_runner.cc:432] Initiating AddMaster RPC to add 127.6.40.124:42743
I20260806 22:47:18.293031 15942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:18.285423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.124:42743 (local address 127.0.0.1:56714)
0806 22:47:18.285878 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:18.285888 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:18.285958 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:18.286123 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:18.286124 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:18.286399 (+ 275us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:18.286491 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:18.286494 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.287402 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.287402 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:18.288323 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:18.288325 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.288785 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.288785 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:18.288888 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:18.289472 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:18.289476 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:18.289818 (+ 342us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:47:18.291862 (+ 2044us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:18.291863 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:18.291864 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:18.292117 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:18.292342 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:18.292342 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:18.292343 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:18.292399 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:18.292667 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:47:18.292667 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:47:18.292699 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 22:47:18.292745 (+ 46us) client_negotiation.cc:240] Negotiation successful
0806 22:47:18.292826 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":434,"mutex_wait_us":3,"spinlock_wait_cycles":10624,"thread_start_us":108,"threads_started":1}
I20260806 22:47:18.293121 15914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:18.285620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56714 (local address 127.6.40.124:42743)
0806 22:47:18.285681 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:18.285683 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:18.285960 (+ 277us) server_negotiation.cc:407] Connection header received
0806 22:47:18.286017 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:18.286018 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:18.286029 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:18.286080 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:18.286574 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.287319 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.288408 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.288672 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.288707 (+ 35us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:18.289981 (+ 1274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:18.289983 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:18.289983 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:18.289990 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:18.291770 (+ 1780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:18.292197 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:18.292199 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:18.292199 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:18.292263 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:18.292466 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:18.292466 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:18.292467 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:18.292598 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:18.292683 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:18.292780 (+ 97us) server_negotiation.cc:299] Negotiation successful
0806 22:47:18.292956 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":41}
I20260806 22:47:18.312170 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 15024
I20260806 22:47:18.325562 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.65:43189
--local_ip_for_outbound_sockets=127.6.40.65
--tserver_master_addrs=127.6.40.126:38077
--webserver_port=38213
--webserver_interface=127.6.40.65
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/krb5kdc/oryx_127.6.40.65.keytab
--principal=oryx/127.6.40.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:34373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.6.40.126:38077,127.6.40.125:42247
--tserver_master_addrs=127.6.40.126:38077,127.6.40.125:42247,127.6.40.124:42743 with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:47:18.325763 15798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:18.285476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52626 (local address 127.6.40.125:42247)
0806 22:47:18.285520 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:18.285522 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:18.285958 (+ 436us) server_negotiation.cc:407] Connection header received
0806 22:47:18.286062 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:18.286063 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:18.286078 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:18.286109 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:18.286732 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.287499 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.288651 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.288984 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.289034 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:18.321788 (+ 32754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:18.321790 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:18.321803 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:18.321815 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:18.324063 (+ 2248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:18.324597 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:18.324599 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:18.324599 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:18.324670 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:18.324982 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:18.324984 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:18.324984 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:18.325172 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:18.325229 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:18.325506 (+ 277us) server_negotiation.cc:299] Negotiation successful
0806 22:47:18.325624 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 22:47:18.326025 14994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:18.285775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48102 (local address 127.6.40.126:38077)
0806 22:47:18.285851 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:18.285853 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:18.286138 (+ 285us) server_negotiation.cc:407] Connection header received
0806 22:47:18.286235 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:18.286236 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:18.286247 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:18.286277 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:18.286742 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.287479 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.288598 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.288930 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.288976 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:18.321744 (+ 32768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:18.321748 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:18.321748 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:18.321760 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:18.324482 (+ 2722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:18.324990 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:18.324991 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:18.324992 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:18.325057 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:18.325339 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:18.325340 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:18.325340 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:18.325510 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:18.325574 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:18.325823 (+ 249us) server_negotiation.cc:299] Negotiation successful
0806 22:47:18.325924 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 22:47:18.326236 15944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:18.285423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.125:42247 (local address 127.0.0.1:52626)
0806 22:47:18.285878 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:18.285888 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:18.285986 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:18.286187 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:18.286188 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:18.286455 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:18.286626 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:18.286629 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.287602 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.287603 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:18.288545 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:18.288548 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.289117 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.289117 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:18.289181 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:18.321010 (+ 31829us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:18.321016 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:18.321635 (+ 619us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:47:18.324202 (+ 2567us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:18.324204 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:18.324205 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:18.324479 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:18.324767 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:18.324768 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:18.324768 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:18.324879 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:18.325291 (+ 412us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:47:18.325292 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:47:18.325353 (+ 61us) client_negotiation.cc:769] Sending connection context
0806 22:47:18.325953 (+ 600us) client_negotiation.cc:240] Negotiation successful
0806 22:47:18.326095 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":98,"threads_started":1}
I20260806 22:47:18.326435 15941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:18.285743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:38077 (local address 127.0.0.1:48102)
0806 22:47:18.286081 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:18.286090 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:18.286162 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:18.286393 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:18.286394 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:18.286600 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:18.286656 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:18.286658 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.287577 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.287578 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:18.288517 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:18.288518 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.289063 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.289064 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:18.289141 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:18.320768 (+ 31627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:18.320776 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:18.321554 (+ 778us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:47:18.324597 (+ 3043us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:18.324598 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:18.324598 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:18.324879 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:18.325156 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:18.325156 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:18.325157 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:18.325224 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:18.325639 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:47:18.325639 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:47:18.325687 (+ 48us) client_negotiation.cc:769] Sending connection context
0806 22:47:18.326158 (+ 471us) client_negotiation.cc:240] Negotiation successful
0806 22:47:18.326264 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"mutex_wait_us":40,"thread_start_us":74,"threads_started":1}
Aug 06 22:47:18 dist-test-slave-zh2d krb5kdc[14880](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056437, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.40.126@KRBTEST.COM for oryx/127.6.40.124@KRBTEST.COM
I20260806 22:47:18.339170 15914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:18.330166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56722 (local address 127.6.40.124:42743)
0806 22:47:18.330207 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:18.330209 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:18.330216 (+ 7us) server_negotiation.cc:407] Connection header received
0806 22:47:18.330251 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:18.330252 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:18.330261 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:18.330315 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:18.330751 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.331494 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.332701 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.333036 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.333119 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:18.335891 (+ 2772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:18.335892 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:18.335893 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:18.335900 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:18.337760 (+ 1860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:18.338250 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:18.338251 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:18.338252 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:18.338312 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:18.338583 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:18.338584 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:18.338584 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:18.338712 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:18.338778 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:18.338993 (+ 215us) server_negotiation.cc:299] Negotiation successful
0806 22:47:18.339072 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 22:47:18.339473 15696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:18.329968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.124:42743 (local address 127.0.0.1:56722)
0806 22:47:18.330031 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:18.330043 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:18.330088 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:18.330360 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:18.330362 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:18.330580 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:18.330650 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:18.330652 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.331593 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.331595 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:18.332597 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:18.332601 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.333140 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.333141 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:18.333241 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:18.333697 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:18.333703 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:18.335776 (+ 2073us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:47:18.337866 (+ 2090us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:18.337868 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:18.337868 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:18.338156 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:18.338397 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:18.338398 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:18.338399 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:18.338493 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:18.338793 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:47:18.338794 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:47:18.338856 (+ 62us) client_negotiation.cc:769] Sending connection context
0806 22:47:18.338960 (+ 104us) client_negotiation.cc:240] Negotiation successful
0806 22:47:18.339115 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260806 22:47:18.340257 14938 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.6.40.124:42743: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "cca9c82ca527425abfcddb8d93dd4b64"
member_type: NON_VOTER
last_known_addr {
host: "127.6.40.124"
port: 42743
}
attrs {
promote: true
}
}
dest_uuid: "eb7a8590eeb54daab448b3b04d3dfacf"
cas_config_opid_index: 5
I20260806 22:47:18.340523 14938 consensus_queue.cc:237] T 00000000000000000000000000000000 P eb7a8590eeb54daab448b3b04d3dfacf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "eb7a8590eeb54daab448b3b04d3dfacf" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 38077 } } peers { permanent_uuid: "0016a76e8ad5476995c85a46a1fdcb6b" member_type: NON_VOTER last_known_addr { host: "127.6.40.125" port: 42247 } attrs { promote: true } } peers { permanent_uuid: "cca9c82ca527425abfcddb8d93dd4b64" member_type: NON_VOTER last_known_addr { host: "127.6.40.124" port: 42743 } attrs { promote: true } }
I20260806 22:47:18.341054 15698 raft_consensus.cc:2932] T 00000000000000000000000000000000 P eb7a8590eeb54daab448b3b04d3dfacf [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER cca9c82ca527425abfcddb8d93dd4b64 (127.6.40.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "eb7a8590eeb54daab448b3b04d3dfacf" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 38077 } } peers { permanent_uuid: "0016a76e8ad5476995c85a46a1fdcb6b" member_type: NON_VOTER last_known_addr { host: "127.6.40.125" port: 42247 } attrs { promote: true } } peers { permanent_uuid: "cca9c82ca527425abfcddb8d93dd4b64" member_type: NON_VOTER last_known_addr { host: "127.6.40.124" port: 42743 } attrs { promote: true } } }
I20260806 22:47:18.342142 15698 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.6.40.124:42743
I20260806 22:47:18.342217 15698 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb7a8590eeb54daab448b3b04d3dfacf [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "eb7a8590eeb54daab448b3b04d3dfacf" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "eb7a8590eeb54daab448b3b04d3dfacf" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 38077 } } peers { permanent_uuid: "0016a76e8ad5476995c85a46a1fdcb6b" member_type: NON_VOTER last_known_addr { host: "127.6.40.125" port: 42247 } attrs { promote: true } } peers { permanent_uuid: "cca9c82ca527425abfcddb8d93dd4b64" member_type: NON_VOTER last_known_addr { host: "127.6.40.124" port: 42743 } attrs { promote: true } } }
I20260806 22:47:18.342293 15698 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb7a8590eeb54daab448b3b04d3dfacf [sys.catalog]: This master's current role is: LEADER
I20260806 22:47:18.342638 15819 master.cc:537] Master@127.6.40.124:42743 shutting down...
I20260806 22:47:18.342811 15874 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cca9c82ca527425abfcddb8d93dd4b64 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:47:18.343881 15874 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cca9c82ca527425abfcddb8d93dd4b64 [term 1 FOLLOWER]: Refusing update from remote peer eb7a8590eeb54daab448b3b04d3dfacf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 22:47:18.344272 15697 consensus_queue.cc:1035] T 00000000000000000000000000000000 P eb7a8590eeb54daab448b3b04d3dfacf [LEADER]: Connected to new peer: Peer: permanent_uuid: "cca9c82ca527425abfcddb8d93dd4b64" member_type: NON_VOTER last_known_addr { host: "127.6.40.124" port: 42743 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 22:47:18.345072 14920 consensus_peers.cc:487] T 00000000000000000000000000000000 P eb7a8590eeb54daab448b3b04d3dfacf -> Peer cca9c82ca527425abfcddb8d93dd4b64 (127.6.40.124:42743): Couldn't send request to peer cca9c82ca527425abfcddb8d93dd4b64. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260806 22:47:18.349013 15798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:18.341092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52636 (local address 127.6.40.125:42247)
0806 22:47:18.341148 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:18.341150 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:18.344611 (+ 3461us) server_negotiation.cc:407] Connection header received
0806 22:47:18.344727 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:18.344728 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:18.344758 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:18.344800 (+ 42us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:47:18.345295 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.346095 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.347762 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.348391 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.348441 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:18.348522 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:18.348880 (+ 358us) server_negotiation.cc:299] Negotiation successful
0806 22:47:18.348937 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 22:47:18.349397 15696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:18.341012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.125:42247 (local address 127.0.0.1:52636)
0806 22:47:18.344549 (+ 3537us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:18.344561 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:18.344644 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:18.344906 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:18.344907 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:18.344909 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:47:18.345063 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:18.345067 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.346444 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.346445 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:18.347647 (+ 1202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:18.347650 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.348584 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.348585 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:18.348750 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:18.348775 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 22:47:18.349284 (+ 509us) client_negotiation.cc:240] Negotiation successful
0806 22:47:18.349333 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3521}
I20260806 22:47:18.349610 15819 raft_consensus.cc:2229] T 00000000000000000000000000000000 P cca9c82ca527425abfcddb8d93dd4b64 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:47:18.349685 15757 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0016a76e8ad5476995c85a46a1fdcb6b [term 1 LEARNER]: Refusing update from remote peer eb7a8590eeb54daab448b3b04d3dfacf: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 22:47:18.349705 15819 raft_consensus.cc:2258] T 00000000000000000000000000000000 P cca9c82ca527425abfcddb8d93dd4b64 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:47:18.349759 15819 tablet_replica.cc:330] T 00000000000000000000000000000000 P cca9c82ca527425abfcddb8d93dd4b64: stopping tablet replica
I20260806 22:47:18.349938 15697 consensus_queue.cc:1035] T 00000000000000000000000000000000 P eb7a8590eeb54daab448b3b04d3dfacf [LEADER]: Connected to new peer: Peer: permanent_uuid: "0016a76e8ad5476995c85a46a1fdcb6b" member_type: NON_VOTER last_known_addr { host: "127.6.40.125" port: 42247 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260806 22:47:18.350582 15757 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0016a76e8ad5476995c85a46a1fdcb6b [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER cca9c82ca527425abfcddb8d93dd4b64 (127.6.40.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "eb7a8590eeb54daab448b3b04d3dfacf" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 38077 } } peers { permanent_uuid: "0016a76e8ad5476995c85a46a1fdcb6b" member_type: NON_VOTER last_known_addr { host: "127.6.40.125" port: 42247 } attrs { promote: true } } peers { permanent_uuid: "cca9c82ca527425abfcddb8d93dd4b64" member_type: NON_VOTER last_known_addr { host: "127.6.40.124" port: 42743 } attrs { promote: true } } }
I20260806 22:47:18.352134 15799 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0016a76e8ad5476995c85a46a1fdcb6b [sys.catalog]: SysCatalogTable state changed. Reason: New leader eb7a8590eeb54daab448b3b04d3dfacf. Latest consensus state: current_term: 1 leader_uuid: "eb7a8590eeb54daab448b3b04d3dfacf" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "eb7a8590eeb54daab448b3b04d3dfacf" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 38077 } } peers { permanent_uuid: "0016a76e8ad5476995c85a46a1fdcb6b" member_type: NON_VOTER last_known_addr { host: "127.6.40.125" port: 42247 } attrs { promote: true } } peers { permanent_uuid: "cca9c82ca527425abfcddb8d93dd4b64" member_type: NON_VOTER last_known_addr { host: "127.6.40.124" port: 42743 } attrs { promote: true } } }
I20260806 22:47:18.352239 15799 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0016a76e8ad5476995c85a46a1fdcb6b [sys.catalog]: This master's current role is: LEARNER
I20260806 22:47:18.352288 15799 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0016a76e8ad5476995c85a46a1fdcb6b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "eb7a8590eeb54daab448b3b04d3dfacf" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "eb7a8590eeb54daab448b3b04d3dfacf" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 38077 } } peers { permanent_uuid: "0016a76e8ad5476995c85a46a1fdcb6b" member_type: NON_VOTER last_known_addr { host: "127.6.40.125" port: 42247 } attrs { promote: true } } peers { permanent_uuid: "cca9c82ca527425abfcddb8d93dd4b64" member_type: NON_VOTER last_known_addr { host: "127.6.40.124" port: 42743 } attrs { promote: true } } }
I20260806 22:47:18.352335 15799 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0016a76e8ad5476995c85a46a1fdcb6b [sys.catalog]: This master's current role is: LEARNER
I20260806 22:47:18.352702 15697 raft_consensus.cc:1050] T 00000000000000000000000000000000 P eb7a8590eeb54daab448b3b04d3dfacf: attempting to promote NON_VOTER 0016a76e8ad5476995c85a46a1fdcb6b to VOTER
I20260806 22:47:18.352952 15697 consensus_queue.cc:237] T 00000000000000000000000000000000 P eb7a8590eeb54daab448b3b04d3dfacf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "eb7a8590eeb54daab448b3b04d3dfacf" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 38077 } } peers { permanent_uuid: "0016a76e8ad5476995c85a46a1fdcb6b" member_type: VOTER last_known_addr { host: "127.6.40.125" port: 42247 } attrs { promote: false } } peers { permanent_uuid: "cca9c82ca527425abfcddb8d93dd4b64" member_type: NON_VOTER last_known_addr { host: "127.6.40.124" port: 42743 } attrs { promote: true } }
I20260806 22:47:18.353593 15757 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0016a76e8ad5476995c85a46a1fdcb6b [term 1 LEARNER]: Refusing update from remote peer eb7a8590eeb54daab448b3b04d3dfacf: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260806 22:47:18.353802 15697 consensus_queue.cc:1035] T 00000000000000000000000000000000 P eb7a8590eeb54daab448b3b04d3dfacf [LEADER]: Connected to new peer: Peer: permanent_uuid: "0016a76e8ad5476995c85a46a1fdcb6b" member_type: VOTER last_known_addr { host: "127.6.40.125" port: 42247 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260806 22:47:18.354224 15819 master.cc:552] Master@127.6.40.124:42743 shutdown complete.
I20260806 22:47:18.354269 15819 master_runner.cc:305] Clearing existing system tablet
I20260806 22:47:18.354400 15819 fs_manager.cc:359] Metadata directory not provided
I20260806 22:47:18.354449 15819 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:47:18.354660 15698 raft_consensus.cc:2932] T 00000000000000000000000000000000 P eb7a8590eeb54daab448b3b04d3dfacf [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 0016a76e8ad5476995c85a46a1fdcb6b (127.6.40.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "eb7a8590eeb54daab448b3b04d3dfacf" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 38077 } } peers { permanent_uuid: "0016a76e8ad5476995c85a46a1fdcb6b" member_type: VOTER last_known_addr { host: "127.6.40.125" port: 42247 } attrs { promote: false } } peers { permanent_uuid: "cca9c82ca527425abfcddb8d93dd4b64" member_type: NON_VOTER last_known_addr { host: "127.6.40.124" port: 42743 } attrs { promote: true } } }
I20260806 22:47:18.355086 15696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:18.354268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.124:42743 (local address 127.0.0.1:56736)
0806 22:47:18.355008 (+ 740us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:18.355056 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.124:42743: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":720}
I20260806 22:47:18.355443 15696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:18.355291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.124:42743 (local address 127.0.0.1:56752)
0806 22:47:18.355392 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:18.355416 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.124:42743: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":69,"threads_started":1}
W20260806 22:47:18.355489 14920 consensus_peers.cc:487] T 00000000000000000000000000000000 P eb7a8590eeb54daab448b3b04d3dfacf -> Peer cca9c82ca527425abfcddb8d93dd4b64 (127.6.40.124:42743): Couldn't send request to peer cca9c82ca527425abfcddb8d93dd4b64. Status: Network error: Client connection negotiation failed: client connection to 127.6.40.124:42743: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 22:47:18.355554 15698 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb7a8590eeb54daab448b3b04d3dfacf [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "eb7a8590eeb54daab448b3b04d3dfacf" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "eb7a8590eeb54daab448b3b04d3dfacf" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 38077 } } peers { permanent_uuid: "0016a76e8ad5476995c85a46a1fdcb6b" member_type: VOTER last_known_addr { host: "127.6.40.125" port: 42247 } attrs { promote: false } } peers { permanent_uuid: "cca9c82ca527425abfcddb8d93dd4b64" member_type: NON_VOTER last_known_addr { host: "127.6.40.124" port: 42743 } attrs { promote: true } } }
I20260806 22:47:18.355630 15698 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb7a8590eeb54daab448b3b04d3dfacf [sys.catalog]: This master's current role is: LEADER
I20260806 22:47:18.355700 15819 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:47:18.355680 15698 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb7a8590eeb54daab448b3b04d3dfacf [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "eb7a8590eeb54daab448b3b04d3dfacf" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "eb7a8590eeb54daab448b3b04d3dfacf" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 38077 } } peers { permanent_uuid: "0016a76e8ad5476995c85a46a1fdcb6b" member_type: VOTER last_known_addr { host: "127.6.40.125" port: 42247 } attrs { promote: false } } peers { permanent_uuid: "cca9c82ca527425abfcddb8d93dd4b64" member_type: NON_VOTER last_known_addr { host: "127.6.40.124" port: 42743 } attrs { promote: true } } }
I20260806 22:47:18.355738 15698 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb7a8590eeb54daab448b3b04d3dfacf [sys.catalog]: This master's current role is: LEADER
I20260806 22:47:18.356314 15757 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0016a76e8ad5476995c85a46a1fdcb6b [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 0016a76e8ad5476995c85a46a1fdcb6b (127.6.40.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "eb7a8590eeb54daab448b3b04d3dfacf" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 38077 } } peers { permanent_uuid: "0016a76e8ad5476995c85a46a1fdcb6b" member_type: VOTER last_known_addr { host: "127.6.40.125" port: 42247 } attrs { promote: false } } peers { permanent_uuid: "cca9c82ca527425abfcddb8d93dd4b64" member_type: NON_VOTER last_known_addr { host: "127.6.40.124" port: 42743 } attrs { promote: true } } }
I20260806 22:47:18.356983 15954 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:18.357090 15819 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:47:18.357131 15819 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 22:47:18.357152 15819 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/master-2/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/master-2/wal
uuid: "cca9c82ca527425abfcddb8d93dd4b64"
format_stamp: "Formatted at 2026-08-06 22:47:18 on dist-test-slave-zh2d"
server_key: "4b9451f51f12cf73d480e409e78a1a96"
server_key_iv: "6331bdf9c05e17707277d1bc8400c999"
server_key_version: "encryptionkey@0"
I20260806 22:47:18.357282 15799 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0016a76e8ad5476995c85a46a1fdcb6b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "eb7a8590eeb54daab448b3b04d3dfacf" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "eb7a8590eeb54daab448b3b04d3dfacf" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 38077 } } peers { permanent_uuid: "0016a76e8ad5476995c85a46a1fdcb6b" member_type: VOTER last_known_addr { host: "127.6.40.125" port: 42247 } attrs { promote: false } } peers { permanent_uuid: "cca9c82ca527425abfcddb8d93dd4b64" member_type: NON_VOTER last_known_addr { host: "127.6.40.124" port: 42743 } attrs { promote: true } } }
I20260806 22:47:18.357354 15799 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0016a76e8ad5476995c85a46a1fdcb6b [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:47:18.357940 15819 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P cca9c82ca527425abfcddb8d93dd4b64: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:47:18.359336 15819 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P cca9c82ca527425abfcddb8d93dd4b64: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 22:47:18.359411 15819 log.cc:1196] T 00000000000000000000000000000000 P cca9c82ca527425abfcddb8d93dd4b64: Deleting WAL directory at /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260806 22:47:18.359627 15819 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P cca9c82ca527425abfcddb8d93dd4b64: Deleting consensus metadata
I20260806 22:47:18.359754 15819 master_runner.cc:315] Copying system tablet from 127.6.40.126:38077
I20260806 22:47:18.360404 15819 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P cca9c82ca527425abfcddb8d93dd4b64: tablet copy: Beginning tablet copy session from remote peer at address 127.6.40.126:38077
I20260806 22:47:18.368510 15963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:18.360708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:38077 (local address 127.0.0.1:48114)
0806 22:47:18.360953 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:18.360964 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:18.361031 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:18.361169 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:18.361170 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:18.361429 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:18.361493 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:18.361496 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.362471 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.362472 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:18.363377 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:18.363380 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.363856 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.363857 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:18.363940 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:18.364901 (+ 961us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:18.364906 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:18.365331 (+ 425us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:47:18.367327 (+ 1996us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:18.367328 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:18.367329 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:18.367558 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:18.367799 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:18.367800 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:18.367801 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:18.367866 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:18.368178 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:47:18.368179 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:47:18.368227 (+ 48us) client_negotiation.cc:769] Sending connection context
0806 22:47:18.368301 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 22:47:18.368409 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":60,"threads_started":1}
I20260806 22:47:18.368544 14994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:18.360716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48114 (local address 127.6.40.126:38077)
0806 22:47:18.360761 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:18.360763 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:18.361037 (+ 274us) server_negotiation.cc:407] Connection header received
0806 22:47:18.361088 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:18.361089 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:18.361100 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:18.361151 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:18.361577 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.362357 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.363467 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.363759 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.363832 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:18.365418 (+ 1586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:18.365419 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:18.365420 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:18.365427 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:18.367227 (+ 1800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:18.367640 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:18.367641 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:18.367642 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:18.367718 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:18.367945 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:18.367946 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:18.367947 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:18.368095 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:18.368185 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:18.368353 (+ 168us) server_negotiation.cc:299] Negotiation successful
0806 22:47:18.368439 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 22:47:18.369346 14968 tablet_copy_service.cc:140] P eb7a8590eeb54daab448b3b04d3dfacf: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer cca9c82ca527425abfcddb8d93dd4b64 ({username='oryx', principal='oryx/127.6.40.124@KRBTEST.COM'} at 127.0.0.1:48114)
I20260806 22:47:18.369401 14968 tablet_copy_service.cc:161] P eb7a8590eeb54daab448b3b04d3dfacf: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer cca9c82ca527425abfcddb8d93dd4b64 at {username='oryx', principal='oryx/127.6.40.124@KRBTEST.COM'} at 127.0.0.1:48114: session id = cca9c82ca527425abfcddb8d93dd4b64-00000000000000000000000000000000
I20260806 22:47:18.370007 14968 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P eb7a8590eeb54daab448b3b04d3dfacf: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 22:47:18.370514 15819 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 22:47:18.371829 15819 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P cca9c82ca527425abfcddb8d93dd4b64: tablet copy: Starting download of 0 data blocks...
I20260806 22:47:18.371965 15819 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P cca9c82ca527425abfcddb8d93dd4b64: tablet copy: Starting download of 1 WAL segments...
I20260806 22:47:18.372896 15819 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P cca9c82ca527425abfcddb8d93dd4b64: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 22:47:18.373888 14968 tablet_copy_service.cc:342] P eb7a8590eeb54daab448b3b04d3dfacf: Request end of tablet copy session cca9c82ca527425abfcddb8d93dd4b64-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.6.40.124@KRBTEST.COM'} at 127.0.0.1:48114
I20260806 22:47:18.373942 14968 tablet_copy_service.cc:433] P eb7a8590eeb54daab448b3b04d3dfacf: ending tablet copy session cca9c82ca527425abfcddb8d93dd4b64-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer cca9c82ca527425abfcddb8d93dd4b64
I20260806 22:47:18.376971 15819 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:47:18.378216 15966 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:18.381781 15967 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:18.382028 15969 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:18.382077 15819 server_base.cc:1034] running on GCE node
I20260806 22:47:18.382395 15819 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:47:18.382550 15819 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:47:18.383690 15819 hybrid_clock.cc:648] HybridClock initialized: now 1786056438383700 us; error 47 us; skew 500 ppm
Aug 06 22:47:18 dist-test-slave-zh2d krb5kdc[14880](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056438, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.40.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:47:18.385334 15819 init.cc:377] Logged in from keytab as oryx/127.6.40.124@KRBTEST.COM (short username oryx)
I20260806 22:47:18.385622 15819 webserver.cc:460] Webserver started at http://127.6.40.124:33529/ using document root <none> and password file <none>
I20260806 22:47:18.385751 15819 fs_manager.cc:359] Metadata directory not provided
I20260806 22:47:18.385790 15819 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:47:18.386598 15819 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:47:18.387228 15977 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:18.387342 15819 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:47:18.387380 15819 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/master-2/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/master-2/wal
uuid: "cca9c82ca527425abfcddb8d93dd4b64"
format_stamp: "Formatted at 2026-08-06 22:47:18 on dist-test-slave-zh2d"
server_key: "4b9451f51f12cf73d480e409e78a1a96"
server_key_iv: "6331bdf9c05e17707277d1bc8400c999"
server_key_version: "encryptionkey@0"
I20260806 22:47:18.387442 15819 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:47:18.396755 15819 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:47:18.398617 15819 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:47:18.401882 15819 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.124:42743
I20260806 22:47:18.402333 15819 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/master-2/data/info.pb
I20260806 22:47:18.403352 16043 sys_catalog.cc:263] Verifying existing consensus state
I20260806 22:47:18.403879 16043 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cca9c82ca527425abfcddb8d93dd4b64: Bootstrap starting.
I20260806 22:47:18.404980 16042 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.124:42743 every 8 connection(s)
I20260806 22:47:18.407249 16043 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cca9c82ca527425abfcddb8d93dd4b64: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 22:47:18.407512 16043 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cca9c82ca527425abfcddb8d93dd4b64: Bootstrap complete.
I20260806 22:47:18.407753 16043 raft_consensus.cc:348] T 00000000000000000000000000000000 P cca9c82ca527425abfcddb8d93dd4b64 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "eb7a8590eeb54daab448b3b04d3dfacf" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 38077 } } peers { permanent_uuid: "0016a76e8ad5476995c85a46a1fdcb6b" member_type: VOTER last_known_addr { host: "127.6.40.125" port: 42247 } attrs { promote: false } } peers { permanent_uuid: "cca9c82ca527425abfcddb8d93dd4b64" member_type: NON_VOTER last_known_addr { host: "127.6.40.124" port: 42743 } attrs { promote: true } }
I20260806 22:47:18.407860 16043 raft_consensus.cc:729] T 00000000000000000000000000000000 P cca9c82ca527425abfcddb8d93dd4b64 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cca9c82ca527425abfcddb8d93dd4b64, State: Initialized, Role: LEARNER
I20260806 22:47:18.407909 16043 consensus_queue.cc:260] T 00000000000000000000000000000000 P cca9c82ca527425abfcddb8d93dd4b64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "eb7a8590eeb54daab448b3b04d3dfacf" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 38077 } } peers { permanent_uuid: "0016a76e8ad5476995c85a46a1fdcb6b" member_type: VOTER last_known_addr { host: "127.6.40.125" port: 42247 } attrs { promote: false } } peers { permanent_uuid: "cca9c82ca527425abfcddb8d93dd4b64" member_type: NON_VOTER last_known_addr { host: "127.6.40.124" port: 42743 } attrs { promote: true } }
I20260806 22:47:18.408113 16043 sys_catalog.cc:564] T 00000000000000000000000000000000 P cca9c82ca527425abfcddb8d93dd4b64 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:47:18.408131 16049 sys_catalog.cc:455] T 00000000000000000000000000000000 P cca9c82ca527425abfcddb8d93dd4b64 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "eb7a8590eeb54daab448b3b04d3dfacf" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 38077 } } peers { permanent_uuid: "0016a76e8ad5476995c85a46a1fdcb6b" member_type: VOTER last_known_addr { host: "127.6.40.125" port: 42247 } attrs { promote: false } } peers { permanent_uuid: "cca9c82ca527425abfcddb8d93dd4b64" member_type: NON_VOTER last_known_addr { host: "127.6.40.124" port: 42743 } attrs { promote: true } } }
I20260806 22:47:18.408357 16049 sys_catalog.cc:458] T 00000000000000000000000000000000 P cca9c82ca527425abfcddb8d93dd4b64 [sys.catalog]: This master's current role is: LEARNER
I20260806 22:47:18.409754 16061 catalog_manager.cc:1259] Loaded cluster ID: 16746f209dcb48eb906c4d27294225e8
I20260806 22:47:18.409862 16061 catalog_manager.cc:1552] T 00000000000000000000000000000000 P cca9c82ca527425abfcddb8d93dd4b64: loading cluster ID for follower catalog manager: success
I20260806 22:47:18.411118 16061 catalog_manager.cc:1574] T 00000000000000000000000000000000 P cca9c82ca527425abfcddb8d93dd4b64: acquiring CA information for follower catalog manager: success
I20260806 22:47:18.411434 16061 catalog_manager.cc:1602] T 00000000000000000000000000000000 P cca9c82ca527425abfcddb8d93dd4b64: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260806 22:47:18.440125 15945 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:47:18.440340 15945 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:47:18.440366 15945 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:47:18.441962 15945 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:47:18.442026 15945 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:47:18.442062 15945 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.65
I20260806 22:47:18.444154 15945 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:34373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/krb5kdc/oryx_127.6.40.65.keytab
--principal=oryx/127.6.40.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.65:43189
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.40.65
--webserver_port=38213
--tserver_master_addrs=127.6.40.126:38077,127.6.40.125:42247,127.6.40.124:42743
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15945
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.65
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:47:18.444410 15945 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:47:18.444657 15945 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:47:18.450078 16078 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:18.450169 16075 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:18.450415 16073 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:18.450493 15945 server_base.cc:1034] running on GCE node
I20260806 22:47:18.450699 15945 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:47:18.451005 15945 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:47:18.452193 15945 hybrid_clock.cc:648] HybridClock initialized: now 1786056438452178 us; error 28 us; skew 500 ppm
Aug 06 22:47:18 dist-test-slave-zh2d krb5kdc[14880](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056438, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.40.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:47:18.454963 15945 init.cc:377] Logged in from keytab as oryx/127.6.40.65@KRBTEST.COM (short username oryx)
I20260806 22:47:18.455554 15945 webserver.cc:460] Webserver started at http://127.6.40.65:38213/ using document root <none> and password file <none>
I20260806 22:47:18.455862 15945 fs_manager.cc:359] Metadata directory not provided
I20260806 22:47:18.455945 15945 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:47:18.457484 15945 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260806 22:47:18.458158 16090 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:18.458397 15945 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:47:18.458500 15945 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/ts-0/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/ts-0/wal
uuid: "35a7f06767bb4b81999ee9fa87d12aa3"
format_stamp: "Formatted at 2026-08-06 22:47:17 on dist-test-slave-zh2d"
server_key: "31dd1440e2bd4eac04444def40941b32"
server_key_iv: "aa3e0aa194adcba88785fa374dd7c6f0"
server_key_version: "encryptionkey@0"
I20260806 22:47:18.458952 15945 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:47:18.469293 15945 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:47:18.471128 15945 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:47:18.471247 15945 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:47:18.471442 15945 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:47:18.471776 15945 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:47:18.471812 15945 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:18.471833 15945 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:47:18.471886 15945 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:18.486439 15945 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.65:43189
I20260806 22:47:18.486485 16246 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.65:43189 every 8 connection(s)
I20260806 22:47:18.486927 15945 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
I20260806 22:47:18.494025 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 15945
I20260806 22:47:18.495026 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 15211
Aug 06 22:47:18 dist-test-slave-zh2d krb5kdc[14880](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056438, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.40.65@KRBTEST.COM for oryx/127.6.40.124@KRBTEST.COM
Aug 06 22:47:18 dist-test-slave-zh2d krb5kdc[14880](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056438, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.40.65@KRBTEST.COM for oryx/127.6.40.125@KRBTEST.COM
Aug 06 22:47:18 dist-test-slave-zh2d krb5kdc[14880](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056438, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.40.65@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM
I20260806 22:47:18.502935 14994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:18.487972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.65:34961 (local address 127.6.40.126:38077)
0806 22:47:18.488031 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:18.488034 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:18.488877 (+ 843us) server_negotiation.cc:407] Connection header received
0806 22:47:18.489598 (+ 721us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:18.489600 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:18.489612 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:18.489646 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:18.490921 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.491433 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.493343 (+ 1910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.493666 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.493730 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:18.498609 (+ 4879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:18.498611 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:18.498612 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:18.498621 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:18.500650 (+ 2029us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:18.501346 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:18.501347 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:18.501348 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:18.501429 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:18.501762 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:18.501763 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:18.501763 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:18.501958 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:18.502045 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:18.502658 (+ 613us) server_negotiation.cc:299] Negotiation successful
0806 22:47:18.502774 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 22:47:18.502965 16256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:18.488052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.65:49897 (local address 127.6.40.124:42743)
0806 22:47:18.488234 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:18.488236 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:18.488685 (+ 449us) server_negotiation.cc:407] Connection header received
0806 22:47:18.489707 (+ 1022us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:18.489709 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:18.489723 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:18.489847 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:18.490955 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.491925 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.493180 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.493485 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.493555 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:18.498684 (+ 5129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:18.498686 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:18.498687 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:18.498696 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:18.501281 (+ 2585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:18.501781 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:18.501783 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:18.501783 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:18.501860 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:18.502127 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:18.502129 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:18.502129 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:18.502348 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:18.502440 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:18.502730 (+ 290us) server_negotiation.cc:299] Negotiation successful
0806 22:47:18.502850 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":87,"threads_started":1}
I20260806 22:47:18.503767 16255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:18.487859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.124:42743 (local address 127.6.40.65:49897)
0806 22:47:18.488560 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:18.488569 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:18.489448 (+ 879us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:18.489883 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:18.489884 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:18.490480 (+ 596us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:18.490838 (+ 358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:18.490839 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.492001 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.492002 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:18.493088 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:18.493091 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.493576 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.493577 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:18.493860 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:18.494611 (+ 751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:18.494622 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:18.498560 (+ 3938us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:47:18.501392 (+ 2832us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:18.501393 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:18.501394 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:18.501681 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:18.501949 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:18.501950 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:18.501951 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:18.502030 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:18.502462 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:47:18.502463 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:47:18.502522 (+ 59us) client_negotiation.cc:769] Sending connection context
0806 22:47:18.502596 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 22:47:18.503547 (+ 951us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":625,"mutex_wait_us":824,"thread_start_us":211,"threads_started":1}
I20260806 22:47:18.504043 16258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:18.487859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:38077 (local address 127.6.40.65:34961)
0806 22:47:18.488787 (+ 928us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:18.488796 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:18.489476 (+ 680us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:18.490169 (+ 693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:18.490170 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:18.490489 (+ 319us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:18.490813 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:18.490819 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.491659 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.491660 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:18.493262 (+ 1602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:18.493264 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.493823 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.493824 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:18.493933 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:18.494967 (+ 1034us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:18.494972 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:18.498507 (+ 3535us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:47:18.500790 (+ 2283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:18.500810 (+ 20us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:18.500813 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:18.501232 (+ 419us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:18.501513 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:18.501515 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:18.501515 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:18.501666 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:18.502091 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:47:18.502099 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:47:18.502343 (+ 244us) client_negotiation.cc:769] Sending connection context
0806 22:47:18.502595 (+ 252us) client_negotiation.cc:240] Negotiation successful
0806 22:47:18.502844 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":907,"mutex_wait_us":103,"thread_start_us":568,"threads_started":1}
I20260806 22:47:18.504743 16248 heartbeater.cc:347] Connected to a master server at 127.6.40.126:38077
I20260806 22:47:18.504940 16248 heartbeater.cc:464] Registering TS with master...
I20260806 22:47:18.505143 16248 heartbeater.cc:511] Master 127.6.40.126:38077 requested a full tablet report, sending...
I20260806 22:47:18.505653 14938 ts_manager.cc:194] Re-registered known tserver with Master: 35a7f06767bb4b81999ee9fa87d12aa3 (127.6.40.65:43189)
I20260806 22:47:18.506142 14938 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.40.65@KRBTEST.COM'} at 127.6.40.65:34961
I20260806 22:47:18.514865 16247 heartbeater.cc:347] Connected to a master server at 127.6.40.124:42743
I20260806 22:47:18.514973 16247 heartbeater.cc:464] Registering TS with master...
I20260806 22:47:18.515091 16247 heartbeater.cc:511] Master 127.6.40.124:42743 requested a full tablet report, sending...
I20260806 22:47:18.515535 15986 ts_manager.cc:194] Registered new tserver with Master: 35a7f06767bb4b81999ee9fa87d12aa3 (127.6.40.65:43189)
I20260806 22:47:18.518023 16257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:18.487857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.125:42247 (local address 127.6.40.65:49385)
0806 22:47:18.488453 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:18.488568 (+ 115us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:18.489448 (+ 880us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:18.489719 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:18.489723 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:18.490504 (+ 781us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:18.490830 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:18.490833 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.491802 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.491803 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:18.493230 (+ 1427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:18.493233 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.493664 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.493665 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:18.493860 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:18.494907 (+ 1047us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:18.494912 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:18.497983 (+ 3071us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:47:18.516897 (+ 18914us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:18.516899 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:18.516900 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:18.517120 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:18.517360 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:18.517361 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:18.517361 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:18.517418 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:18.517743 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:47:18.517744 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:47:18.517787 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 22:47:18.517832 (+ 45us) client_negotiation.cc:240] Negotiation successful
0806 22:47:18.517942 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":552,"mutex_wait_us":78,"thread_start_us":409,"threads_started":1}
I20260806 22:47:18.518131 15798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:18.488103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.65:49385 (local address 127.6.40.125:42247)
0806 22:47:18.488189 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:18.488191 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:18.488689 (+ 498us) server_negotiation.cc:407] Connection header received
0806 22:47:18.489563 (+ 874us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:18.489565 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:18.489579 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:18.489644 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:18.490948 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.491720 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.493336 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.493583 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.493676 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:18.514703 (+ 21027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:18.514706 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:18.514706 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:18.514719 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:18.516756 (+ 2037us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:18.517219 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:18.517221 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:18.517221 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:18.517294 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:18.517500 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:18.517501 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:18.517501 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:18.517683 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:18.517751 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:18.517891 (+ 140us) server_negotiation.cc:299] Negotiation successful
0806 22:47:18.517993 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 22:47:18.518606 16252 heartbeater.cc:347] Connected to a master server at 127.6.40.125:42247
I20260806 22:47:18.518674 16252 heartbeater.cc:464] Registering TS with master...
I20260806 22:47:18.518731 16252 heartbeater.cc:511] Master 127.6.40.125:42247 requested a full tablet report, sending...
I20260806 22:47:18.519114 15745 ts_manager.cc:194] Registered new tserver with Master: 35a7f06767bb4b81999ee9fa87d12aa3 (127.6.40.65:43189)
I20260806 22:47:18.520123 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.66:34829
--local_ip_for_outbound_sockets=127.6.40.66
--tserver_master_addrs=127.6.40.126:38077
--webserver_port=40053
--webserver_interface=127.6.40.66
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/krb5kdc/oryx_127.6.40.66.keytab
--principal=oryx/127.6.40.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:34373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.6.40.126:38077,127.6.40.125:42247
--tserver_master_addrs=127.6.40.126:38077,127.6.40.125:42247,127.6.40.124:42743 with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:47:18.605545 16259 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:47:18.605751 16259 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:47:18.605775 16259 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:47:18.606961 16259 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:47:18.607002 16259 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:47:18.607028 16259 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.66
I20260806 22:47:18.608721 16259 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:34373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/krb5kdc/oryx_127.6.40.66.keytab
--principal=oryx/127.6.40.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.66:34829
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.40.66
--webserver_port=40053
--tserver_master_addrs=127.6.40.126:38077,127.6.40.125:42247,127.6.40.124:42743
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16259
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.66
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:47:18.609027 16259 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:47:18.609337 16259 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:47:18.614665 16265 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:18.614771 16259 server_base.cc:1034] running on GCE node
W20260806 22:47:18.614674 16267 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:18.614692 16269 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:18.615258 16259 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:47:18.615506 16259 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:47:18.616663 16259 hybrid_clock.cc:648] HybridClock initialized: now 1786056438616651 us; error 27 us; skew 500 ppm
Aug 06 22:47:18 dist-test-slave-zh2d krb5kdc[14880](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056438, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.40.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:47:18.619244 16259 init.cc:377] Logged in from keytab as oryx/127.6.40.66@KRBTEST.COM (short username oryx)
I20260806 22:47:18.619678 16259 webserver.cc:460] Webserver started at http://127.6.40.66:40053/ using document root <none> and password file <none>
I20260806 22:47:18.619907 16259 fs_manager.cc:359] Metadata directory not provided
I20260806 22:47:18.619967 16259 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:47:18.621381 16259 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:47:18.622056 16279 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:18.622264 16259 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:47:18.622360 16259 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/ts-1/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/ts-1/wal
uuid: "d1fec59295474125ae87691d06eaf345"
format_stamp: "Formatted at 2026-08-06 22:47:17 on dist-test-slave-zh2d"
server_key: "376719fcbdf4d4a70eb6cd2e7fa3c5b0"
server_key_iv: "2316a0bb93da27e3bbfc86d97120ff9d"
server_key_version: "encryptionkey@0"
I20260806 22:47:18.622714 16259 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:47:18.631804 16259 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:47:18.633778 16259 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:47:18.633942 16259 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:47:18.634181 16259 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:47:18.634549 16259 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:47:18.634608 16259 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:18.634649 16259 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:47:18.634677 16259 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:18.641465 16259 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.66:34829
I20260806 22:47:18.641599 16434 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.66:34829 every 8 connection(s)
I20260806 22:47:18.642081 16259 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
I20260806 22:47:18.645638 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 16259
I20260806 22:47:18.645838 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 15391
Aug 06 22:47:18 dist-test-slave-zh2d krb5kdc[14880](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056438, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.40.66@KRBTEST.COM for oryx/127.6.40.125@KRBTEST.COM
I20260806 22:47:18.657002 15798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:18.643657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.66:44033 (local address 127.6.40.125:42247)
0806 22:47:18.643710 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:18.643712 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:18.644094 (+ 382us) server_negotiation.cc:407] Connection header received
0806 22:47:18.645314 (+ 1220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:18.645315 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:18.645329 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:18.645364 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:18.646663 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.647769 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.649384 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.649667 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.649707 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:18.652858 (+ 3151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:18.652860 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:18.652861 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:18.652869 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:18.654908 (+ 2039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:18.655545 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:18.655547 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:18.655547 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:18.655611 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:18.655900 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:18.655901 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:18.655901 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:18.656052 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:18.656097 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:18.656533 (+ 436us) server_negotiation.cc:299] Negotiation successful
0806 22:47:18.656657 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 22:47:18.657658 16441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:18.643544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.125:42247 (local address 127.6.40.66:44033)
0806 22:47:18.644031 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:18.644041 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:18.645200 (+ 1159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:18.645542 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:18.645543 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:18.646065 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:18.646527 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:18.646536 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.647907 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.647908 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:18.649290 (+ 1382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:18.649293 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.649788 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.649789 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:18.649994 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:18.650621 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:18.650631 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:18.652729 (+ 2098us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:47:18.655037 (+ 2308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:18.655046 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:18.655048 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:18.655449 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:18.655678 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:18.655679 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:18.655679 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:18.655821 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:18.656145 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:47:18.656152 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:47:18.656320 (+ 168us) client_negotiation.cc:769] Sending connection context
0806 22:47:18.657127 (+ 807us) client_negotiation.cc:240] Negotiation successful
0806 22:47:18.657343 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":189,"threads_started":1}
Aug 06 22:47:18 dist-test-slave-zh2d krb5kdc[14880](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056438, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.40.66@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM
I20260806 22:47:18.667589 16438 heartbeater.cc:347] Connected to a master server at 127.6.40.125:42247
I20260806 22:47:18.667745 16438 heartbeater.cc:464] Registering TS with master...
I20260806 22:47:18.668221 16438 heartbeater.cc:511] Master 127.6.40.125:42247 requested a full tablet report, sending...
I20260806 22:47:18.668824 15745 ts_manager.cc:194] Registered new tserver with Master: d1fec59295474125ae87691d06eaf345 (127.6.40.66:34829)
I20260806 22:47:18.669924 14994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:18.643078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.66:43761 (local address 127.6.40.126:38077)
0806 22:47:18.643129 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:18.643131 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:18.644047 (+ 916us) server_negotiation.cc:407] Connection header received
0806 22:47:18.645313 (+ 1266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:18.645314 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:18.645328 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:18.645364 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:18.653523 (+ 8159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.654333 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.655903 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.656194 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.656235 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:18.666032 (+ 9797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:18.666035 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:18.666035 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:18.666048 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:18.668313 (+ 2265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:18.668826 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:18.668827 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:18.668827 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:18.668926 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:18.669197 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:18.669198 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:18.669198 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:18.669378 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:18.669455 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:18.669690 (+ 235us) server_negotiation.cc:299] Negotiation successful
0806 22:47:18.669812 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 22:47:18.669950 16440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:18.643034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:38077 (local address 127.6.40.66:43761)
0806 22:47:18.643860 (+ 826us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:18.643917 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:18.645200 (+ 1283us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:18.645465 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:18.645466 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:18.653146 (+ 7680us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:18.653384 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:18.653387 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.654597 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.654598 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:18.655764 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:18.655766 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.656313 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.656314 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:18.656467 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:18.657987 (+ 1520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:18.657992 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:18.665870 (+ 7878us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:47:18.668436 (+ 2566us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:18.668438 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:18.668438 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:18.668717 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:18.669023 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:18.669024 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:18.669025 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:18.669102 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:18.669502 (+ 400us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:47:18.669503 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:47:18.669554 (+ 51us) client_negotiation.cc:769] Sending connection context
0806 22:47:18.669637 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 22:47:18.669820 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":738,"thread_start_us":445,"threads_started":1}
I20260806 22:47:18.670389 16436 heartbeater.cc:347] Connected to a master server at 127.6.40.126:38077
I20260806 22:47:18.670490 16436 heartbeater.cc:464] Registering TS with master...
I20260806 22:47:18.670584 16436 heartbeater.cc:511] Master 127.6.40.126:38077 requested a full tablet report, sending...
I20260806 22:47:18.670887 14938 ts_manager.cc:194] Re-registered known tserver with Master: d1fec59295474125ae87691d06eaf345 (127.6.40.66:34829)
I20260806 22:47:18.671460 14938 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.40.66@KRBTEST.COM'} at 127.6.40.66:43761
I20260806 22:47:18.671582 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.67:45455
--local_ip_for_outbound_sockets=127.6.40.67
--tserver_master_addrs=127.6.40.126:38077
--webserver_port=43699
--webserver_interface=127.6.40.67
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/krb5kdc/oryx_127.6.40.67.keytab
--principal=oryx/127.6.40.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:34373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.6.40.126:38077,127.6.40.125:42247
--tserver_master_addrs=127.6.40.126:38077,127.6.40.125:42247,127.6.40.124:42743 with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 22:47:18 dist-test-slave-zh2d krb5kdc[14880](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056438, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.40.66@KRBTEST.COM for oryx/127.6.40.124@KRBTEST.COM
I20260806 22:47:18.675518 16256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:18.643143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.66:39681 (local address 127.6.40.124:42743)
0806 22:47:18.643187 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:18.643189 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:18.644071 (+ 882us) server_negotiation.cc:407] Connection header received
0806 22:47:18.645310 (+ 1239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:18.645312 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:18.645330 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:18.645386 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:18.666228 (+ 20842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.667086 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.668505 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.668900 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.668950 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:18.672511 (+ 3561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:18.672513 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:18.672513 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:18.672522 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:18.674282 (+ 1760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:18.674665 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:18.674667 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:18.674667 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:18.674731 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:18.674958 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:18.674959 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:18.674960 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:18.675076 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:18.675150 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:18.675310 (+ 160us) server_negotiation.cc:299] Negotiation successful
0806 22:47:18.675418 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 22:47:18.675530 16442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:18.643008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.124:42743 (local address 127.6.40.66:39681)
0806 22:47:18.643952 (+ 944us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:18.643965 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:18.645173 (+ 1208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:18.645455 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:18.645460 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:18.666051 (+ 20591us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:18.666108 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:18.666110 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.667206 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.667207 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:18.668401 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:18.668404 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.669081 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.669082 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:18.669145 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:18.669644 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:18.669650 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:18.672415 (+ 2765us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:47:18.674395 (+ 1980us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:18.674396 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:18.674397 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:18.674585 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:18.674808 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:18.674809 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:18.674810 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:18.674874 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:18.675154 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:47:18.675155 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:47:18.675198 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 22:47:18.675283 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 22:47:18.675407 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":860,"thread_start_us":437,"threads_started":1}
I20260806 22:47:18.676085 16435 heartbeater.cc:347] Connected to a master server at 127.6.40.124:42743
I20260806 22:47:18.676142 16435 heartbeater.cc:464] Registering TS with master...
I20260806 22:47:18.676220 16435 heartbeater.cc:511] Master 127.6.40.124:42743 requested a full tablet report, sending...
I20260806 22:47:18.676605 15986 ts_manager.cc:194] Registered new tserver with Master: d1fec59295474125ae87691d06eaf345 (127.6.40.66:34829)
I20260806 22:47:18.741042 15952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:18.737629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.124:42743 (local address 127.0.0.1:56760)
0806 22:47:18.737707 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:18.737722 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:18.737800 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:18.738014 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:18.738014 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:18.738017 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:47:18.738075 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:18.738076 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.739089 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.739089 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:18.739813 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:18.739815 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.740728 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.740729 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:18.740785 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:18.740790 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:47:18.740822 (+ 32us) client_negotiation.cc:240] Negotiation successful
0806 22:47:18.740918 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260806 22:47:18.741029 16256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:18.737742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56760 (local address 127.6.40.124:42743)
0806 22:47:18.737829 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:18.737831 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:18.737846 (+ 15us) server_negotiation.cc:407] Connection header received
0806 22:47:18.737884 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:18.737885 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:18.737918 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:18.738010 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:47:18.738127 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.739000 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.739901 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.740628 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.740733 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:18.740817 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:18.740891 (+ 74us) server_negotiation.cc:299] Negotiation successful
0806 22:47:18.740939 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 22:47:18.741544 16013 raft_consensus.cc:1203] T 00000000000000000000000000000000 P cca9c82ca527425abfcddb8d93dd4b64 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260806 22:47:18.741729 16049 sys_catalog.cc:455] T 00000000000000000000000000000000 P cca9c82ca527425abfcddb8d93dd4b64 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eb7a8590eeb54daab448b3b04d3dfacf. Latest consensus state: current_term: 1 leader_uuid: "eb7a8590eeb54daab448b3b04d3dfacf" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "eb7a8590eeb54daab448b3b04d3dfacf" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 38077 } } peers { permanent_uuid: "0016a76e8ad5476995c85a46a1fdcb6b" member_type: VOTER last_known_addr { host: "127.6.40.125" port: 42247 } attrs { promote: false } } peers { permanent_uuid: "cca9c82ca527425abfcddb8d93dd4b64" member_type: NON_VOTER last_known_addr { host: "127.6.40.124" port: 42743 } attrs { promote: true } } }
I20260806 22:47:18.741904 16049 sys_catalog.cc:458] T 00000000000000000000000000000000 P cca9c82ca527425abfcddb8d93dd4b64 [sys.catalog]: This master's current role is: LEARNER
W20260806 22:47:18.771644 16443 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:47:18.771888 16443 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:47:18.771916 16443 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:47:18.773244 16443 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:47:18.773299 16443 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:47:18.773332 16443 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.67
I20260806 22:47:18.774982 16443 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:34373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/krb5kdc/oryx_127.6.40.67.keytab
--principal=oryx/127.6.40.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.67:45455
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.40.67
--webserver_port=43699
--tserver_master_addrs=127.6.40.126:38077,127.6.40.125:42247,127.6.40.124:42743
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16443
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.67
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:47:18.775202 16443 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:47:18.775413 16443 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:47:18.780766 16454 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:18.780905 16443 server_base.cc:1034] running on GCE node
W20260806 22:47:18.780865 16451 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:18.780766 16450 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:18.781296 16443 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:47:18.781550 16443 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:47:18.782709 16443 hybrid_clock.cc:648] HybridClock initialized: now 1786056438782698 us; error 31 us; skew 500 ppm
Aug 06 22:47:18 dist-test-slave-zh2d krb5kdc[14880](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056438, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.40.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:47:18.785480 16443 init.cc:377] Logged in from keytab as oryx/127.6.40.67@KRBTEST.COM (short username oryx)
I20260806 22:47:18.785948 16443 webserver.cc:460] Webserver started at http://127.6.40.67:43699/ using document root <none> and password file <none>
I20260806 22:47:18.786182 16443 fs_manager.cc:359] Metadata directory not provided
I20260806 22:47:18.786265 16443 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:47:18.787668 16443 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260806 22:47:18.788291 16462 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:18.788509 16443 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 22:47:18.788610 16443 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/ts-2/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/ts-2/wal
uuid: "62e43f48d5464af9bcbd68d8fb9877d4"
format_stamp: "Formatted at 2026-08-06 22:47:17 on dist-test-slave-zh2d"
server_key: "8be2ae534ae39f5ffd3abbe3698cbfbe"
server_key_iv: "8adfcfcfa2f2154f3ed6bc86235d6dfe"
server_key_version: "encryptionkey@0"
I20260806 22:47:18.788988 16443 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:47:18.801780 16443 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:47:18.803766 16443 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:47:18.803933 16443 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:47:18.804172 16443 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:47:18.804504 16443 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:47:18.804559 16443 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:18.804625 16443 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:47:18.804654 16443 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:18.812033 16443 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.67:45455
I20260806 22:47:18.812176 16618 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.67:45455 every 8 connection(s)
I20260806 22:47:18.812501 16443 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
I20260806 22:47:18.818113 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 16443
Aug 06 22:47:18 dist-test-slave-zh2d krb5kdc[14880](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056438, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.40.67@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM
Aug 06 22:47:18 dist-test-slave-zh2d krb5kdc[14880](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056438, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.40.67@KRBTEST.COM for oryx/127.6.40.124@KRBTEST.COM
Aug 06 22:47:18 dist-test-slave-zh2d krb5kdc[14880](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056438, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.40.67@KRBTEST.COM for oryx/127.6.40.125@KRBTEST.COM
I20260806 22:47:18.829080 15798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:18.813577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.67:42129 (local address 127.6.40.125:42247)
0806 22:47:18.813648 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:18.813650 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:18.814289 (+ 639us) server_negotiation.cc:407] Connection header received
0806 22:47:18.815062 (+ 773us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:18.815063 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:18.815077 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:18.815136 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:18.815994 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.816784 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.818881 (+ 2097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.819189 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.819227 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:18.825523 (+ 6296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:18.825525 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:18.825525 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:18.825533 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:18.827254 (+ 1721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:18.827854 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:18.827855 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:18.827855 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:18.827923 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:18.828246 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:18.828247 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:18.828247 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:18.828405 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:18.828495 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:18.828883 (+ 388us) server_negotiation.cc:299] Negotiation successful
0806 22:47:18.828960 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 22:47:18.829423 14994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:18.813383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.67:55337 (local address 127.6.40.126:38077)
0806 22:47:18.813443 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:18.813445 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:18.814253 (+ 808us) server_negotiation.cc:407] Connection header received
0806 22:47:18.815081 (+ 828us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:18.815082 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:18.815095 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:18.815132 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:18.816292 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.816787 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.818921 (+ 2134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.819255 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.819301 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:18.823278 (+ 3977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:18.823280 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:18.823280 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:18.823298 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:18.827864 (+ 4566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:18.828268 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:18.828269 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:18.828269 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:18.828331 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:18.828573 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:18.828575 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:18.828575 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:18.828720 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:18.828783 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:18.829243 (+ 460us) server_negotiation.cc:299] Negotiation successful
0806 22:47:18.829325 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 22:47:18.829993 16623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:18.813393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:38077 (local address 127.6.40.67:55337)
0806 22:47:18.814056 (+ 663us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:18.814114 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:18.814972 (+ 858us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:18.815735 (+ 763us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:18.815736 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:18.816022 (+ 286us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:18.816203 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:18.816205 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.817062 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.817064 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:18.818774 (+ 1710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:18.818777 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.819388 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.819389 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:18.819544 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:18.820264 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:18.820283 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:18.823175 (+ 2892us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:47:18.827961 (+ 4786us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:18.827962 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:18.827963 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:18.828190 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:18.828405 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:18.828406 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:18.828406 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:18.828486 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:18.828817 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:47:18.828830 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:47:18.828895 (+ 65us) client_negotiation.cc:769] Sending connection context
0806 22:47:18.828981 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 22:47:18.829653 (+ 672us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":598,"mutex_wait_us":46,"thread_start_us":540,"threads_started":1}
I20260806 22:47:18.830063 16631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:18.819492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48126 (local address 127.6.40.126:38077)
0806 22:47:18.819929 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:18.819931 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:18.820862 (+ 931us) server_negotiation.cc:407] Connection header received
0806 22:47:18.820956 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:18.820957 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:18.820971 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:18.821081 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:18.822048 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.822841 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.823994 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.824388 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.824463 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:18.826456 (+ 1993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:18.826457 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:18.826458 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:18.826464 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:18.828748 (+ 2284us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:18.829166 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:18.829167 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:18.829167 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:18.829235 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:18.829429 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:18.829430 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:18.829430 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:18.829600 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:18.829692 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:18.829882 (+ 190us) server_negotiation.cc:299] Negotiation successful
0806 22:47:18.829962 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":162,"threads_started":1}
I20260806 22:47:18.830559 16620 heartbeater.cc:347] Connected to a master server at 127.6.40.126:38077
I20260806 22:47:18.830664 16620 heartbeater.cc:464] Registering TS with master...
I20260806 22:47:18.830879 16620 heartbeater.cc:511] Master 127.6.40.126:38077 requested a full tablet report, sending...
I20260806 22:47:18.831331 14940 ts_manager.cc:194] Re-registered known tserver with Master: 62e43f48d5464af9bcbd68d8fb9877d4 (127.6.40.67:45455)
I20260806 22:47:18.831821 14940 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.40.67@KRBTEST.COM'} at 127.6.40.67:55337
I20260806 22:47:18.833086 16624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:18.813393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.125:42247 (local address 127.6.40.67:42129)
0806 22:47:18.814163 (+ 770us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:18.814175 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:18.814948 (+ 773us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:18.815202 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:18.815206 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:18.815586 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:18.815881 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:18.815886 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.817528 (+ 1642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.817530 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:18.818767 (+ 1237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:18.818769 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.819316 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.819317 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:18.819725 (+ 408us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:18.820437 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:18.820442 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:18.825387 (+ 4945us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:47:18.827364 (+ 1977us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:18.827411 (+ 47us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:18.827414 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:18.827764 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:18.828001 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:18.828002 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:18.828002 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:18.828156 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:18.828513 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:47:18.828520 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:47:18.828691 (+ 171us) client_negotiation.cc:769] Sending connection context
0806 22:47:18.829397 (+ 706us) client_negotiation.cc:240] Negotiation successful
0806 22:47:18.832883 (+ 3486us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":714,"mutex_wait_us":3380,"thread_start_us":696,"threads_started":1}
I20260806 22:47:18.833810 16621 heartbeater.cc:347] Connected to a master server at 127.6.40.125:42247
I20260806 22:47:18.833873 16621 heartbeater.cc:464] Registering TS with master...
I20260806 22:47:18.833945 16621 heartbeater.cc:511] Master 127.6.40.125:42247 requested a full tablet report, sending...
I20260806 22:47:18.833957 16256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:18.813493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.67:54425 (local address 127.6.40.124:42743)
0806 22:47:18.813565 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:18.813567 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:18.814363 (+ 796us) server_negotiation.cc:407] Connection header received
0806 22:47:18.815081 (+ 718us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:18.815082 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:18.815101 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:18.815134 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:18.816144 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.816903 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.818876 (+ 1973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.819174 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.819218 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:18.825009 (+ 5791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:18.825010 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:18.825011 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:18.825020 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:18.827332 (+ 2312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:18.832806 (+ 5474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:18.832807 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:18.832807 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:18.832908 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:18.833374 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:18.833375 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:18.833376 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:18.833522 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:18.833572 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:18.833782 (+ 210us) server_negotiation.cc:299] Negotiation successful
0806 22:47:18.833870 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 22:47:18.831782 14938 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48126:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 22:47:18.834275 15745 ts_manager.cc:194] Registered new tserver with Master: 62e43f48d5464af9bcbd68d8fb9877d4 (127.6.40.67:45455)
W20260806 22:47:18.834345 14938 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:47:18.834797 16625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:18.813393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.124:42743 (local address 127.6.40.67:54425)
0806 22:47:18.814244 (+ 851us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:18.814253 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:18.814973 (+ 720us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:18.815302 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:18.815302 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:18.815770 (+ 468us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:18.816031 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:18.816034 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.817076 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.817077 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:18.818766 (+ 1689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:18.818769 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.819318 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.819319 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:18.819538 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:18.820695 (+ 1157us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:18.820700 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:18.823512 (+ 2812us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:47:18.831767 (+ 8255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:18.831768 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:18.831768 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:18.832065 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:18.832992 (+ 927us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:18.832993 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:18.832994 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:18.833071 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:18.833626 (+ 555us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:47:18.833627 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:47:18.833669 (+ 42us) client_negotiation.cc:769] Sending connection context
0806 22:47:18.834372 (+ 703us) client_negotiation.cc:240] Negotiation successful
0806 22:47:18.834565 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":776,"mutex_wait_us":231,"thread_start_us":404,"threads_started":1}
I20260806 22:47:18.835481 15698 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb7a8590eeb54daab448b3b04d3dfacf [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "eb7a8590eeb54daab448b3b04d3dfacf" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "eb7a8590eeb54daab448b3b04d3dfacf" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 38077 } } peers { permanent_uuid: "0016a76e8ad5476995c85a46a1fdcb6b" member_type: VOTER last_known_addr { host: "127.6.40.125" port: 42247 } attrs { promote: false } } peers { permanent_uuid: "cca9c82ca527425abfcddb8d93dd4b64" member_type: NON_VOTER last_known_addr { host: "127.6.40.124" port: 42743 } attrs { promote: true } } }
I20260806 22:47:18.835579 15698 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb7a8590eeb54daab448b3b04d3dfacf [sys.catalog]: This master's current role is: LEADER
I20260806 22:47:18.836452 16619 heartbeater.cc:347] Connected to a master server at 127.6.40.124:42743
I20260806 22:47:18.836506 16619 heartbeater.cc:464] Registering TS with master...
I20260806 22:47:18.836550 16619 heartbeater.cc:511] Master 127.6.40.124:42743 requested a full tablet report, sending...
I20260806 22:47:18.837298 15986 ts_manager.cc:194] Registered new tserver with Master: 62e43f48d5464af9bcbd68d8fb9877d4 (127.6.40.67:45455)
I20260806 22:47:18.837314 16636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:18.820148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52640 (local address 127.6.40.125:42247)
0806 22:47:18.822925 (+ 2777us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:18.822927 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:18.822933 (+ 6us) server_negotiation.cc:407] Connection header received
0806 22:47:18.822955 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:18.822956 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:18.822966 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:18.823981 (+ 1015us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:18.824025 (+ 44us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.824803 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.826208 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.826628 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.827021 (+ 393us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:18.828276 (+ 1255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:18.828277 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:18.828277 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:18.828284 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:18.830384 (+ 2100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:18.830982 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:18.830983 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:18.830984 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:18.831068 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:18.831602 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:18.831602 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:18.831603 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:18.832472 (+ 869us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:18.837077 (+ 4605us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:18.837122 (+ 45us) server_negotiation.cc:299] Negotiation successful
0806 22:47:18.837215 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2755,"thread_start_us":56,"threads_started":1}
I20260806 22:47:18.838114 16639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:18.820577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56766 (local address 127.6.40.124:42743)
0806 22:47:18.821399 (+ 822us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:18.821401 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:18.821409 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:47:18.821431 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:18.821431 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:18.821444 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:18.822186 (+ 742us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:18.822229 (+ 43us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.822785 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.831743 (+ 8958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.832066 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.832124 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:18.834042 (+ 1918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:18.834044 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:18.834044 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:18.834051 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:18.836032 (+ 1981us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:18.836429 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:18.836430 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:18.836430 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:18.836493 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:18.836700 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:18.836701 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:18.836701 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:18.836868 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:18.836968 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:18.837841 (+ 873us) server_negotiation.cc:299] Negotiation successful
0806 22:47:18.837930 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":801,"thread_start_us":72,"threads_started":1}
I20260806 22:47:18.839079 15698 raft_consensus.cc:1050] T 00000000000000000000000000000000 P eb7a8590eeb54daab448b3b04d3dfacf: attempting to promote NON_VOTER cca9c82ca527425abfcddb8d93dd4b64 to VOTER
I20260806 22:47:18.839195 15698 consensus_queue.cc:237] T 00000000000000000000000000000000 P eb7a8590eeb54daab448b3b04d3dfacf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "eb7a8590eeb54daab448b3b04d3dfacf" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 38077 } } peers { permanent_uuid: "0016a76e8ad5476995c85a46a1fdcb6b" member_type: VOTER last_known_addr { host: "127.6.40.125" port: 42247 } attrs { promote: false } } peers { permanent_uuid: "cca9c82ca527425abfcddb8d93dd4b64" member_type: VOTER last_known_addr { host: "127.6.40.124" port: 42743 } attrs { promote: false } }
I20260806 22:47:18.839623 15757 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0016a76e8ad5476995c85a46a1fdcb6b [term 1 FOLLOWER]: Refusing update from remote peer eb7a8590eeb54daab448b3b04d3dfacf: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20260806 22:47:18.839869 16012 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cca9c82ca527425abfcddb8d93dd4b64 [term 1 LEARNER]: Refusing update from remote peer eb7a8590eeb54daab448b3b04d3dfacf: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20260806 22:47:18.840104 15697 consensus_queue.cc:1035] T 00000000000000000000000000000000 P eb7a8590eeb54daab448b3b04d3dfacf [LEADER]: Connected to new peer: Peer: permanent_uuid: "0016a76e8ad5476995c85a46a1fdcb6b" member_type: VOTER last_known_addr { host: "127.6.40.125" port: 42247 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260806 22:47:18.840209 15697 consensus_queue.cc:1035] T 00000000000000000000000000000000 P eb7a8590eeb54daab448b3b04d3dfacf [LEADER]: Connected to new peer: Peer: permanent_uuid: "cca9c82ca527425abfcddb8d93dd4b64" member_type: VOTER last_known_addr { host: "127.6.40.124" port: 42743 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260806 22:47:18.840714 15697 raft_consensus.cc:2932] T 00000000000000000000000000000000 P eb7a8590eeb54daab448b3b04d3dfacf [term 1 LEADER]: Committing config change with OpId 1.10: config changed from index 7 to 10, cca9c82ca527425abfcddb8d93dd4b64 (127.6.40.124) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "eb7a8590eeb54daab448b3b04d3dfacf" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 38077 } } peers { permanent_uuid: "0016a76e8ad5476995c85a46a1fdcb6b" member_type: VOTER last_known_addr { host: "127.6.40.125" port: 42247 } attrs { promote: false } } peers { permanent_uuid: "cca9c82ca527425abfcddb8d93dd4b64" member_type: VOTER last_known_addr { host: "127.6.40.124" port: 42743 } attrs { promote: false } } }
I20260806 22:47:18.841334 16012 raft_consensus.cc:2932] T 00000000000000000000000000000000 P cca9c82ca527425abfcddb8d93dd4b64 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 7 to 10, cca9c82ca527425abfcddb8d93dd4b64 (127.6.40.124) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "eb7a8590eeb54daab448b3b04d3dfacf" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 38077 } } peers { permanent_uuid: "0016a76e8ad5476995c85a46a1fdcb6b" member_type: VOTER last_known_addr { host: "127.6.40.125" port: 42247 } attrs { promote: false } } peers { permanent_uuid: "cca9c82ca527425abfcddb8d93dd4b64" member_type: VOTER last_known_addr { host: "127.6.40.124" port: 42743 } attrs { promote: false } } }
I20260806 22:47:18.841943 15697 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb7a8590eeb54daab448b3b04d3dfacf [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "eb7a8590eeb54daab448b3b04d3dfacf" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "eb7a8590eeb54daab448b3b04d3dfacf" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 38077 } } peers { permanent_uuid: "0016a76e8ad5476995c85a46a1fdcb6b" member_type: VOTER last_known_addr { host: "127.6.40.125" port: 42247 } attrs { promote: false } } peers { permanent_uuid: "cca9c82ca527425abfcddb8d93dd4b64" member_type: VOTER last_known_addr { host: "127.6.40.124" port: 42743 } attrs { promote: false } } }
I20260806 22:47:18.842007 15949 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb7a8590eeb54daab448b3b04d3dfacf [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "eb7a8590eeb54daab448b3b04d3dfacf" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "eb7a8590eeb54daab448b3b04d3dfacf" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 38077 } } peers { permanent_uuid: "0016a76e8ad5476995c85a46a1fdcb6b" member_type: VOTER last_known_addr { host: "127.6.40.125" port: 42247 } attrs { promote: false } } peers { permanent_uuid: "cca9c82ca527425abfcddb8d93dd4b64" member_type: VOTER last_known_addr { host: "127.6.40.124" port: 42743 } attrs { promote: false } } }
I20260806 22:47:18.842101 15949 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb7a8590eeb54daab448b3b04d3dfacf [sys.catalog]: This master's current role is: LEADER
I20260806 22:47:18.842118 15698 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb7a8590eeb54daab448b3b04d3dfacf [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "eb7a8590eeb54daab448b3b04d3dfacf" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "eb7a8590eeb54daab448b3b04d3dfacf" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 38077 } } peers { permanent_uuid: "0016a76e8ad5476995c85a46a1fdcb6b" member_type: VOTER last_known_addr { host: "127.6.40.125" port: 42247 } attrs { promote: false } } peers { permanent_uuid: "cca9c82ca527425abfcddb8d93dd4b64" member_type: VOTER last_known_addr { host: "127.6.40.124" port: 42743 } attrs { promote: false } } }
I20260806 22:47:18.842017 15697 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb7a8590eeb54daab448b3b04d3dfacf [sys.catalog]: This master's current role is: LEADER
I20260806 22:47:18.842172 15698 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb7a8590eeb54daab448b3b04d3dfacf [sys.catalog]: This master's current role is: LEADER
I20260806 22:47:18.842355 15757 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0016a76e8ad5476995c85a46a1fdcb6b [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 7 to 10, cca9c82ca527425abfcddb8d93dd4b64 (127.6.40.124) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "eb7a8590eeb54daab448b3b04d3dfacf" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 38077 } } peers { permanent_uuid: "0016a76e8ad5476995c85a46a1fdcb6b" member_type: VOTER last_known_addr { host: "127.6.40.125" port: 42247 } attrs { promote: false } } peers { permanent_uuid: "cca9c82ca527425abfcddb8d93dd4b64" member_type: VOTER last_known_addr { host: "127.6.40.124" port: 42743 } attrs { promote: false } } }
I20260806 22:47:18.842701 16049 sys_catalog.cc:455] T 00000000000000000000000000000000 P cca9c82ca527425abfcddb8d93dd4b64 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "eb7a8590eeb54daab448b3b04d3dfacf" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "eb7a8590eeb54daab448b3b04d3dfacf" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 38077 } } peers { permanent_uuid: "0016a76e8ad5476995c85a46a1fdcb6b" member_type: VOTER last_known_addr { host: "127.6.40.125" port: 42247 } attrs { promote: false } } peers { permanent_uuid: "cca9c82ca527425abfcddb8d93dd4b64" member_type: VOTER last_known_addr { host: "127.6.40.124" port: 42743 } attrs { promote: false } } }
I20260806 22:47:18.842794 16049 sys_catalog.cc:458] T 00000000000000000000000000000000 P cca9c82ca527425abfcddb8d93dd4b64 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:47:18.843382 15948 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0016a76e8ad5476995c85a46a1fdcb6b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "eb7a8590eeb54daab448b3b04d3dfacf" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "eb7a8590eeb54daab448b3b04d3dfacf" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 38077 } } peers { permanent_uuid: "0016a76e8ad5476995c85a46a1fdcb6b" member_type: VOTER last_known_addr { host: "127.6.40.125" port: 42247 } attrs { promote: false } } peers { permanent_uuid: "cca9c82ca527425abfcddb8d93dd4b64" member_type: VOTER last_known_addr { host: "127.6.40.124" port: 42743 } attrs { promote: false } } }
I20260806 22:47:18.843477 15948 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0016a76e8ad5476995c85a46a1fdcb6b [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:47:18.845695 16654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:18.840126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.67:45455 (local address 127.0.0.1:58322)
0806 22:47:18.840930 (+ 804us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:18.840941 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:18.840992 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:18.841406 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:18.841408 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:18.841410 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:47:18.841459 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:18.841461 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.843186 (+ 1725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.843187 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:18.844362 (+ 1175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:18.844364 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.845338 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.845339 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:18.845529 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:18.845538 (+ 9us) client_negotiation.cc:769] Sending connection context
0806 22:47:18.845599 (+ 61us) client_negotiation.cc:240] Negotiation successful
0806 22:47:18.845637 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":756,"thread_start_us":165,"threads_started":1}
I20260806 22:47:18.845781 16658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:18.840940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58322 (local address 127.6.40.67:45455)
0806 22:47:18.841217 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:18.841222 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:18.841232 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:47:18.841296 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:18.841298 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:18.841315 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:18.841407 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:47:18.841531 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.843114 (+ 1583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.844434 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.845258 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.845341 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:18.845511 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:18.845616 (+ 105us) server_negotiation.cc:299] Negotiation successful
0806 22:47:18.845657 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":218,"thread_start_us":139,"threads_started":1}
I20260806 22:47:18.846630 16532 tablet_service.cc:1467] Processing CreateTablet for tablet d61a031a45674a39bbe82e0310349d1a (DEFAULT_TABLE table=test-table [id=c2b11781369e4aef84d3324f47bd1396]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:47:18.847016 16532 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d61a031a45674a39bbe82e0310349d1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:47:18.848150 15696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:18.840037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.65:43189 (local address 127.0.0.1:47200)
0806 22:47:18.840330 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:18.840344 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:18.840388 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:18.844472 (+ 4084us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:18.844474 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:18.844476 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:47:18.844515 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:18.844516 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.845848 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.845850 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:18.847000 (+ 1150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:18.847003 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.847958 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.847959 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:18.848017 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:18.848020 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:47:18.848046 (+ 26us) client_negotiation.cc:240] Negotiation successful
0806 22:47:18.848081 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274}
I20260806 22:47:18.848562 16656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:18.840571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47200 (local address 127.6.40.65:43189)
0806 22:47:18.844274 (+ 3703us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:18.844278 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:18.844288 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:47:18.844329 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:18.844331 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:18.844345 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:18.844458 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:47:18.844565 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.845705 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.847110 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.847860 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.848315 (+ 455us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:18.848404 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:18.848439 (+ 35us) server_negotiation.cc:299] Negotiation successful
0806 22:47:18.848479 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3629,"thread_start_us":158,"threads_started":1}
I20260806 22:47:18.849391 15952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:18.840101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.66:34829 (local address 127.0.0.1:47958)
0806 22:47:18.840421 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:18.840431 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:18.840479 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:18.844134 (+ 3655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:18.844135 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:18.844137 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:47:18.844170 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:18.844171 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.847124 (+ 2953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.847125 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:18.848147 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:18.848149 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.849140 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.849141 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:18.849203 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:18.849206 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:47:18.849275 (+ 69us) client_negotiation.cc:240] Negotiation successful
0806 22:47:18.849330 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306}
I20260806 22:47:18.849502 16655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:18.840555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47958 (local address 127.6.40.66:34829)
0806 22:47:18.843976 (+ 3421us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:18.843980 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:18.843990 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:47:18.844026 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:18.844028 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:18.844041 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:18.844112 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:47:18.844269 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.847023 (+ 2754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.848250 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.849044 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.849141 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:18.849304 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:18.849342 (+ 38us) server_negotiation.cc:299] Negotiation successful
0806 22:47:18.849388 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3371,"thread_start_us":173,"threads_started":1}
I20260806 22:47:18.849400 16158 tablet_service.cc:1467] Processing CreateTablet for tablet d61a031a45674a39bbe82e0310349d1a (DEFAULT_TABLE table=test-table [id=c2b11781369e4aef84d3324f47bd1396]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:47:18.849823 16158 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d61a031a45674a39bbe82e0310349d1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:47:18.850291 16338 tablet_service.cc:1467] Processing CreateTablet for tablet d61a031a45674a39bbe82e0310349d1a (DEFAULT_TABLE table=test-table [id=c2b11781369e4aef84d3324f47bd1396]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:47:18.850656 16338 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d61a031a45674a39bbe82e0310349d1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:47:18.850980 16663 tablet_bootstrap.cc:492] T d61a031a45674a39bbe82e0310349d1a P 62e43f48d5464af9bcbd68d8fb9877d4: Bootstrap starting.
I20260806 22:47:18.851686 16663 tablet_bootstrap.cc:654] T d61a031a45674a39bbe82e0310349d1a P 62e43f48d5464af9bcbd68d8fb9877d4: Neither blocks nor log segments found. Creating new log.
I20260806 22:47:18.852041 16663 log.cc:824] T d61a031a45674a39bbe82e0310349d1a P 62e43f48d5464af9bcbd68d8fb9877d4: Log is configured to *not* fsync() on all Append() calls
I20260806 22:47:18.854178 16665 tablet_bootstrap.cc:492] T d61a031a45674a39bbe82e0310349d1a P d1fec59295474125ae87691d06eaf345: Bootstrap starting.
I20260806 22:47:18.854746 16665 tablet_bootstrap.cc:654] T d61a031a45674a39bbe82e0310349d1a P d1fec59295474125ae87691d06eaf345: Neither blocks nor log segments found. Creating new log.
I20260806 22:47:18.855028 16665 log.cc:824] T d61a031a45674a39bbe82e0310349d1a P d1fec59295474125ae87691d06eaf345: Log is configured to *not* fsync() on all Append() calls
I20260806 22:47:18.855228 16667 tablet_bootstrap.cc:492] T d61a031a45674a39bbe82e0310349d1a P 35a7f06767bb4b81999ee9fa87d12aa3: Bootstrap starting.
I20260806 22:47:18.855803 16667 tablet_bootstrap.cc:654] T d61a031a45674a39bbe82e0310349d1a P 35a7f06767bb4b81999ee9fa87d12aa3: Neither blocks nor log segments found. Creating new log.
I20260806 22:47:18.856081 16667 log.cc:824] T d61a031a45674a39bbe82e0310349d1a P 35a7f06767bb4b81999ee9fa87d12aa3: Log is configured to *not* fsync() on all Append() calls
I20260806 22:47:18.857062 16665 tablet_bootstrap.cc:492] T d61a031a45674a39bbe82e0310349d1a P d1fec59295474125ae87691d06eaf345: No bootstrap required, opened a new log
I20260806 22:47:18.857131 16667 tablet_bootstrap.cc:492] T d61a031a45674a39bbe82e0310349d1a P 35a7f06767bb4b81999ee9fa87d12aa3: No bootstrap required, opened a new log
I20260806 22:47:18.857147 16665 ts_tablet_manager.cc:1397] T d61a031a45674a39bbe82e0310349d1a P d1fec59295474125ae87691d06eaf345: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260806 22:47:18.857234 16667 ts_tablet_manager.cc:1397] T d61a031a45674a39bbe82e0310349d1a P 35a7f06767bb4b81999ee9fa87d12aa3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:47:18.859849 16665 raft_consensus.cc:348] T d61a031a45674a39bbe82e0310349d1a P d1fec59295474125ae87691d06eaf345 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35a7f06767bb4b81999ee9fa87d12aa3" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 43189 } } peers { permanent_uuid: "62e43f48d5464af9bcbd68d8fb9877d4" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45455 } } peers { permanent_uuid: "d1fec59295474125ae87691d06eaf345" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 34829 } }
I20260806 22:47:18.860106 16665 raft_consensus.cc:374] T d61a031a45674a39bbe82e0310349d1a P d1fec59295474125ae87691d06eaf345 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:47:18.860158 16665 raft_consensus.cc:729] T d61a031a45674a39bbe82e0310349d1a P d1fec59295474125ae87691d06eaf345 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1fec59295474125ae87691d06eaf345, State: Initialized, Role: FOLLOWER
I20260806 22:47:18.860281 16665 consensus_queue.cc:260] T d61a031a45674a39bbe82e0310349d1a P d1fec59295474125ae87691d06eaf345 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35a7f06767bb4b81999ee9fa87d12aa3" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 43189 } } peers { permanent_uuid: "62e43f48d5464af9bcbd68d8fb9877d4" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45455 } } peers { permanent_uuid: "d1fec59295474125ae87691d06eaf345" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 34829 } }
I20260806 22:47:18.860651 16665 ts_tablet_manager.cc:1428] T d61a031a45674a39bbe82e0310349d1a P d1fec59295474125ae87691d06eaf345: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260806 22:47:18.860738 16436 heartbeater.cc:503] Master 127.6.40.126:38077 was elected leader, sending a full tablet report...
I20260806 22:47:18.861868 16663 tablet_bootstrap.cc:492] T d61a031a45674a39bbe82e0310349d1a P 62e43f48d5464af9bcbd68d8fb9877d4: No bootstrap required, opened a new log
I20260806 22:47:18.861963 16663 ts_tablet_manager.cc:1397] T d61a031a45674a39bbe82e0310349d1a P 62e43f48d5464af9bcbd68d8fb9877d4: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.000s
I20260806 22:47:18.864114 16667 raft_consensus.cc:348] T d61a031a45674a39bbe82e0310349d1a P 35a7f06767bb4b81999ee9fa87d12aa3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35a7f06767bb4b81999ee9fa87d12aa3" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 43189 } } peers { permanent_uuid: "62e43f48d5464af9bcbd68d8fb9877d4" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45455 } } peers { permanent_uuid: "d1fec59295474125ae87691d06eaf345" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 34829 } }
I20260806 22:47:18.864352 16667 raft_consensus.cc:374] T d61a031a45674a39bbe82e0310349d1a P 35a7f06767bb4b81999ee9fa87d12aa3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:47:18.864408 16667 raft_consensus.cc:729] T d61a031a45674a39bbe82e0310349d1a P 35a7f06767bb4b81999ee9fa87d12aa3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35a7f06767bb4b81999ee9fa87d12aa3, State: Initialized, Role: FOLLOWER
I20260806 22:47:18.864590 16667 consensus_queue.cc:260] T d61a031a45674a39bbe82e0310349d1a P 35a7f06767bb4b81999ee9fa87d12aa3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35a7f06767bb4b81999ee9fa87d12aa3" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 43189 } } peers { permanent_uuid: "62e43f48d5464af9bcbd68d8fb9877d4" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45455 } } peers { permanent_uuid: "d1fec59295474125ae87691d06eaf345" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 34829 } }
I20260806 22:47:18.865110 16667 ts_tablet_manager.cc:1428] T d61a031a45674a39bbe82e0310349d1a P 35a7f06767bb4b81999ee9fa87d12aa3: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260806 22:47:18.865459 16248 heartbeater.cc:503] Master 127.6.40.126:38077 was elected leader, sending a full tablet report...
I20260806 22:47:18.865613 16663 raft_consensus.cc:348] T d61a031a45674a39bbe82e0310349d1a P 62e43f48d5464af9bcbd68d8fb9877d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35a7f06767bb4b81999ee9fa87d12aa3" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 43189 } } peers { permanent_uuid: "62e43f48d5464af9bcbd68d8fb9877d4" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45455 } } peers { permanent_uuid: "d1fec59295474125ae87691d06eaf345" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 34829 } }
I20260806 22:47:18.865775 16663 raft_consensus.cc:374] T d61a031a45674a39bbe82e0310349d1a P 62e43f48d5464af9bcbd68d8fb9877d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:47:18.865835 16663 raft_consensus.cc:729] T d61a031a45674a39bbe82e0310349d1a P 62e43f48d5464af9bcbd68d8fb9877d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62e43f48d5464af9bcbd68d8fb9877d4, State: Initialized, Role: FOLLOWER
I20260806 22:47:18.865949 16663 consensus_queue.cc:260] T d61a031a45674a39bbe82e0310349d1a P 62e43f48d5464af9bcbd68d8fb9877d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35a7f06767bb4b81999ee9fa87d12aa3" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 43189 } } peers { permanent_uuid: "62e43f48d5464af9bcbd68d8fb9877d4" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45455 } } peers { permanent_uuid: "d1fec59295474125ae87691d06eaf345" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 34829 } }
I20260806 22:47:18.866842 16663 ts_tablet_manager.cc:1428] T d61a031a45674a39bbe82e0310349d1a P 62e43f48d5464af9bcbd68d8fb9877d4: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260806 22:47:18.867250 16620 heartbeater.cc:503] Master 127.6.40.126:38077 was elected leader, sending a full tablet report...
W20260806 22:47:18.894309 16439 tablet.cc:2369] T d61a031a45674a39bbe82e0310349d1a P d1fec59295474125ae87691d06eaf345: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:47:18.984442 16671 raft_consensus.cc:482] T d61a031a45674a39bbe82e0310349d1a P d1fec59295474125ae87691d06eaf345 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:47:18.984570 16671 raft_consensus.cc:504] T d61a031a45674a39bbe82e0310349d1a P d1fec59295474125ae87691d06eaf345 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35a7f06767bb4b81999ee9fa87d12aa3" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 43189 } } peers { permanent_uuid: "62e43f48d5464af9bcbd68d8fb9877d4" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45455 } } peers { permanent_uuid: "d1fec59295474125ae87691d06eaf345" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 34829 } }
I20260806 22:47:18.985075 16671 leader_election.cc:288] T d61a031a45674a39bbe82e0310349d1a P d1fec59295474125ae87691d06eaf345 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 35a7f06767bb4b81999ee9fa87d12aa3 (127.6.40.65:43189), 62e43f48d5464af9bcbd68d8fb9877d4 (127.6.40.67:45455)
I20260806 22:47:18.988493 16656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:18.985415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.66:51415 (local address 127.6.40.65:43189)
0806 22:47:18.985488 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:18.985490 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:18.985502 (+ 12us) server_negotiation.cc:407] Connection header received
0806 22:47:18.985608 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:18.985609 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:18.985624 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:18.985703 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:47:18.985958 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.986692 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.987699 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.988072 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.988157 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:18.988169 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:18.988374 (+ 205us) server_negotiation.cc:299] Negotiation successful
0806 22:47:18.988416 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 22:47:18.988493 16442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:18.985372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.65:43189 (local address 127.6.40.66:51415)
0806 22:47:18.985443 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:18.985457 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:18.985515 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:18.985709 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:18.985711 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:18.985713 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:47:18.985859 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:18.985861 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.986778 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.986780 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:18.987586 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:18.987588 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.988155 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.988157 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:18.988289 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:18.988293 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:47:18.988367 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 22:47:18.988424 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260806 22:47:18.988878 16183 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d61a031a45674a39bbe82e0310349d1a" candidate_uuid: "d1fec59295474125ae87691d06eaf345" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35a7f06767bb4b81999ee9fa87d12aa3" is_pre_election: true
W20260806 22:47:18.989058 16253 tablet.cc:2369] T d61a031a45674a39bbe82e0310349d1a P 35a7f06767bb4b81999ee9fa87d12aa3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:47:18.989077 16183 raft_consensus.cc:2454] T d61a031a45674a39bbe82e0310349d1a P 35a7f06767bb4b81999ee9fa87d12aa3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1fec59295474125ae87691d06eaf345 in term 0.
I20260806 22:47:18.989174 16658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:18.985630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.66:41863 (local address 127.6.40.67:45455)
0806 22:47:18.985712 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:18.985715 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:18.985727 (+ 12us) server_negotiation.cc:407] Connection header received
0806 22:47:18.985772 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:18.985773 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:18.985787 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:18.985877 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:47:18.986083 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.986877 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.988075 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.988707 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.988793 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:18.988805 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:18.989048 (+ 243us) server_negotiation.cc:299] Negotiation successful
0806 22:47:18.989097 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 22:47:18.989243 16440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:18.985546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.67:45455 (local address 127.6.40.66:41863)
0806 22:47:18.985603 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:18.985614 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:18.985676 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:18.985881 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:18.985882 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:18.985884 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:47:18.985986 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:18.985988 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.986983 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.986984 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:18.987981 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:18.987983 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:18.988794 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:18.988795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:18.988954 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:18.988959 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:47:18.989029 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 22:47:18.989069 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260806 22:47:18.989329 16280 leader_election.cc:302] T d61a031a45674a39bbe82e0310349d1a P d1fec59295474125ae87691d06eaf345 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35a7f06767bb4b81999ee9fa87d12aa3, d1fec59295474125ae87691d06eaf345; no voters:
I20260806 22:47:18.989460 16671 raft_consensus.cc:2781] T d61a031a45674a39bbe82e0310349d1a P d1fec59295474125ae87691d06eaf345 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:47:18.989535 16671 raft_consensus.cc:482] T d61a031a45674a39bbe82e0310349d1a P d1fec59295474125ae87691d06eaf345 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:47:18.989563 16671 raft_consensus.cc:3037] T d61a031a45674a39bbe82e0310349d1a P d1fec59295474125ae87691d06eaf345 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:47:18.990180 16561 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d61a031a45674a39bbe82e0310349d1a" candidate_uuid: "d1fec59295474125ae87691d06eaf345" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62e43f48d5464af9bcbd68d8fb9877d4" is_pre_election: true
I20260806 22:47:18.990296 16671 raft_consensus.cc:504] T d61a031a45674a39bbe82e0310349d1a P d1fec59295474125ae87691d06eaf345 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35a7f06767bb4b81999ee9fa87d12aa3" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 43189 } } peers { permanent_uuid: "62e43f48d5464af9bcbd68d8fb9877d4" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45455 } } peers { permanent_uuid: "d1fec59295474125ae87691d06eaf345" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 34829 } }
I20260806 22:47:18.990372 16561 raft_consensus.cc:2454] T d61a031a45674a39bbe82e0310349d1a P 62e43f48d5464af9bcbd68d8fb9877d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1fec59295474125ae87691d06eaf345 in term 0.
I20260806 22:47:18.990520 16671 leader_election.cc:288] T d61a031a45674a39bbe82e0310349d1a P d1fec59295474125ae87691d06eaf345 [CANDIDATE]: Term 1 election: Requested vote from peers 35a7f06767bb4b81999ee9fa87d12aa3 (127.6.40.65:43189), 62e43f48d5464af9bcbd68d8fb9877d4 (127.6.40.67:45455)
I20260806 22:47:18.990574 16183 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d61a031a45674a39bbe82e0310349d1a" candidate_uuid: "d1fec59295474125ae87691d06eaf345" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35a7f06767bb4b81999ee9fa87d12aa3"
I20260806 22:47:18.990658 16183 raft_consensus.cc:3037] T d61a031a45674a39bbe82e0310349d1a P 35a7f06767bb4b81999ee9fa87d12aa3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:47:18.990676 16561 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d61a031a45674a39bbe82e0310349d1a" candidate_uuid: "d1fec59295474125ae87691d06eaf345" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62e43f48d5464af9bcbd68d8fb9877d4"
I20260806 22:47:18.990756 16561 raft_consensus.cc:3037] T d61a031a45674a39bbe82e0310349d1a P 62e43f48d5464af9bcbd68d8fb9877d4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:47:18.991600 16183 raft_consensus.cc:2454] T d61a031a45674a39bbe82e0310349d1a P 35a7f06767bb4b81999ee9fa87d12aa3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1fec59295474125ae87691d06eaf345 in term 1.
I20260806 22:47:18.991771 16280 leader_election.cc:302] T d61a031a45674a39bbe82e0310349d1a P d1fec59295474125ae87691d06eaf345 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35a7f06767bb4b81999ee9fa87d12aa3, d1fec59295474125ae87691d06eaf345; no voters:
I20260806 22:47:18.991911 16671 raft_consensus.cc:2781] T d61a031a45674a39bbe82e0310349d1a P d1fec59295474125ae87691d06eaf345 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:47:18.992100 16671 raft_consensus.cc:686] T d61a031a45674a39bbe82e0310349d1a P d1fec59295474125ae87691d06eaf345 [term 1 LEADER]: Becoming Leader. State: Replica: d1fec59295474125ae87691d06eaf345, State: Running, Role: LEADER
I20260806 22:47:18.992165 16561 raft_consensus.cc:2454] T d61a031a45674a39bbe82e0310349d1a P 62e43f48d5464af9bcbd68d8fb9877d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1fec59295474125ae87691d06eaf345 in term 1.
I20260806 22:47:18.992197 16671 consensus_queue.cc:237] T d61a031a45674a39bbe82e0310349d1a P d1fec59295474125ae87691d06eaf345 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35a7f06767bb4b81999ee9fa87d12aa3" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 43189 } } peers { permanent_uuid: "62e43f48d5464af9bcbd68d8fb9877d4" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45455 } } peers { permanent_uuid: "d1fec59295474125ae87691d06eaf345" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 34829 } }
I20260806 22:47:18.993080 14938 catalog_manager.cc:5495] T d61a031a45674a39bbe82e0310349d1a P d1fec59295474125ae87691d06eaf345 reported cstate change: term changed from 0 to 1, leader changed from <none> to d1fec59295474125ae87691d06eaf345 (127.6.40.66). New cstate: current_term: 1 leader_uuid: "d1fec59295474125ae87691d06eaf345" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35a7f06767bb4b81999ee9fa87d12aa3" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 43189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62e43f48d5464af9bcbd68d8fb9877d4" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45455 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1fec59295474125ae87691d06eaf345" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 34829 } health_report { overall_health: HEALTHY } } }
I20260806 22:47:19.027552 16655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:19.024694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47966 (local address 127.6.40.66:34829)
0806 22:47:19.024769 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:19.024771 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:19.024778 (+ 7us) server_negotiation.cc:407] Connection header received
0806 22:47:19.024818 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:19.024819 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:19.024902 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:19.024991 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:47:19.025166 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:19.025841 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:19.026718 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:19.026929 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:19.027004 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:19.027200 (+ 196us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:47:19.027311 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:47:19.027418 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:19.027438 (+ 20us) server_negotiation.cc:299] Negotiation successful
0806 22:47:19.027461 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 22:47:19.029968 16183 raft_consensus.cc:1261] T d61a031a45674a39bbe82e0310349d1a P 35a7f06767bb4b81999ee9fa87d12aa3 [term 1 FOLLOWER]: Refusing update from remote peer d1fec59295474125ae87691d06eaf345: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:47:19.030031 16561 raft_consensus.cc:1261] T d61a031a45674a39bbe82e0310349d1a P 62e43f48d5464af9bcbd68d8fb9877d4 [term 1 FOLLOWER]: Refusing update from remote peer d1fec59295474125ae87691d06eaf345: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:47:19.030417 16671 consensus_queue.cc:1035] T d61a031a45674a39bbe82e0310349d1a P d1fec59295474125ae87691d06eaf345 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35a7f06767bb4b81999ee9fa87d12aa3" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 43189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:47:19.030618 16675 consensus_queue.cc:1035] T d61a031a45674a39bbe82e0310349d1a P d1fec59295474125ae87691d06eaf345 [LEADER]: Connected to new peer: Peer: permanent_uuid: "62e43f48d5464af9bcbd68d8fb9877d4" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45455 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:47:19.037751 14938 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48126:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:47:19.037937 14938 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48126:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:47:19.039831 14938 catalog_manager.cc:5782] T 00000000000000000000000000000000 P eb7a8590eeb54daab448b3b04d3dfacf: Sending DeleteTablet for 3 replicas of tablet d61a031a45674a39bbe82e0310349d1a
I20260806 22:47:19.040297 16158 tablet_service.cc:1514] Processing DeleteTablet for tablet d61a031a45674a39bbe82e0310349d1a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:47:19 UTC) from {username='oryx', principal='oryx/127.6.40.126@KRBTEST.COM'} at 127.0.0.1:47200
I20260806 22:47:19.040297 16338 tablet_service.cc:1514] Processing DeleteTablet for tablet d61a031a45674a39bbe82e0310349d1a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:47:19 UTC) from {username='oryx', principal='oryx/127.6.40.126@KRBTEST.COM'} at 127.0.0.1:47958
I20260806 22:47:19.040517 16692 tablet_replica.cc:330] T d61a031a45674a39bbe82e0310349d1a P 35a7f06767bb4b81999ee9fa87d12aa3: stopping tablet replica
I20260806 22:47:19.040629 16692 raft_consensus.cc:2229] T d61a031a45674a39bbe82e0310349d1a P 35a7f06767bb4b81999ee9fa87d12aa3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:47:19.040733 16692 raft_consensus.cc:2258] T d61a031a45674a39bbe82e0310349d1a P 35a7f06767bb4b81999ee9fa87d12aa3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:47:19.040779 16532 tablet_service.cc:1514] Processing DeleteTablet for tablet d61a031a45674a39bbe82e0310349d1a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:47:19 UTC) from {username='oryx', principal='oryx/127.6.40.126@KRBTEST.COM'} at 127.0.0.1:58322
I20260806 22:47:19.041378 16695 tablet_replica.cc:330] T d61a031a45674a39bbe82e0310349d1a P d1fec59295474125ae87691d06eaf345: stopping tablet replica
I20260806 22:47:19.041487 16695 raft_consensus.cc:2229] T d61a031a45674a39bbe82e0310349d1a P d1fec59295474125ae87691d06eaf345 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:47:19.041549 16692 ts_tablet_manager.cc:1905] T d61a031a45674a39bbe82e0310349d1a P 35a7f06767bb4b81999ee9fa87d12aa3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:47:19.041625 16695 raft_consensus.cc:2258] T d61a031a45674a39bbe82e0310349d1a P d1fec59295474125ae87691d06eaf345 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:47:19.042266 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 14899
I20260806 22:47:19.042264 16695 ts_tablet_manager.cc:1905] T d61a031a45674a39bbe82e0310349d1a P d1fec59295474125ae87691d06eaf345: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:47:19.045444 16696 tablet_replica.cc:330] T d61a031a45674a39bbe82e0310349d1a P 62e43f48d5464af9bcbd68d8fb9877d4: stopping tablet replica
I20260806 22:47:19.045558 16696 raft_consensus.cc:2229] T d61a031a45674a39bbe82e0310349d1a P 62e43f48d5464af9bcbd68d8fb9877d4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:47:19.045651 16696 raft_consensus.cc:2258] T d61a031a45674a39bbe82e0310349d1a P 62e43f48d5464af9bcbd68d8fb9877d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:47:19.052551 16695 ts_tablet_manager.cc:1918] T d61a031a45674a39bbe82e0310349d1a P d1fec59295474125ae87691d06eaf345: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:47:19.052651 16695 log.cc:1196] T d61a031a45674a39bbe82e0310349d1a P d1fec59295474125ae87691d06eaf345: Deleting WAL directory at /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/ts-1/wal/wals/d61a031a45674a39bbe82e0310349d1a
I20260806 22:47:19.053014 16695 ts_tablet_manager.cc:1939] T d61a031a45674a39bbe82e0310349d1a P d1fec59295474125ae87691d06eaf345: Deleting consensus metadata
I20260806 22:47:19.053431 16696 ts_tablet_manager.cc:1905] T d61a031a45674a39bbe82e0310349d1a P 62e43f48d5464af9bcbd68d8fb9877d4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:47:19.056028 16696 ts_tablet_manager.cc:1918] T d61a031a45674a39bbe82e0310349d1a P 62e43f48d5464af9bcbd68d8fb9877d4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:47:19.056145 16696 log.cc:1196] T d61a031a45674a39bbe82e0310349d1a P 62e43f48d5464af9bcbd68d8fb9877d4: Deleting WAL directory at /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/ts-2/wal/wals/d61a031a45674a39bbe82e0310349d1a
I20260806 22:47:19.056564 16696 ts_tablet_manager.cc:1939] T d61a031a45674a39bbe82e0310349d1a P 62e43f48d5464af9bcbd68d8fb9877d4: Deleting consensus metadata
I20260806 22:47:19.057235 16692 ts_tablet_manager.cc:1918] T d61a031a45674a39bbe82e0310349d1a P 35a7f06767bb4b81999ee9fa87d12aa3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:47:19.057346 16692 log.cc:1196] T d61a031a45674a39bbe82e0310349d1a P 35a7f06767bb4b81999ee9fa87d12aa3: Deleting WAL directory at /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/ts-0/wal/wals/d61a031a45674a39bbe82e0310349d1a
I20260806 22:47:19.057662 16692 ts_tablet_manager.cc:1939] T d61a031a45674a39bbe82e0310349d1a P 35a7f06767bb4b81999ee9fa87d12aa3: Deleting consensus metadata
W20260806 22:47:19.058189 16465 connection.cc:535] server connection from 127.0.0.1:58322 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260806 22:47:19.058151 16280 connection.cc:535] server connection from 127.0.0.1:47958 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260806 22:47:19.058413 16094 connection.cc:535] server connection from 127.0.0.1:47200 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 22:47:19.068388 16639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:19.041216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56774 (local address 127.6.40.124:42743)
0806 22:47:19.041465 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:19.041467 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:19.058292 (+ 16825us) server_negotiation.cc:407] Connection header received
0806 22:47:19.058441 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:19.058443 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:19.058464 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:19.058535 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:19.059460 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:19.060265 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:19.061503 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:19.061908 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:19.062009 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:19.063788 (+ 1779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:19.063790 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:19.063812 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:19.063822 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:19.066306 (+ 2484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:19.067086 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:19.067087 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:19.067088 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:19.067161 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:19.067395 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:19.067396 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:19.067396 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:19.067542 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:19.067649 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:19.068081 (+ 432us) server_negotiation.cc:299] Negotiation successful
0806 22:47:19.068209 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":222}
I20260806 22:47:19.069399 16636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:19.060725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52642 (local address 127.6.40.125:42247)
0806 22:47:19.060900 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:19.060902 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:19.061347 (+ 445us) server_negotiation.cc:407] Connection header received
0806 22:47:19.061451 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:19.061452 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:19.061467 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:19.061498 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:19.062475 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:19.063251 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:19.064312 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:19.064630 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:19.064673 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:19.066168 (+ 1495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:19.066169 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:19.066169 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:19.066177 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:19.067945 (+ 1768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:19.068425 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:19.068426 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:19.068426 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:19.068484 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:19.068714 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:19.068716 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:19.068716 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:19.068894 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:19.069007 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:19.069206 (+ 199us) server_negotiation.cc:299] Negotiation successful
0806 22:47:19.069302 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144}
I20260806 22:47:19.070827 16256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:19.060915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56788 (local address 127.6.40.124:42743)
0806 22:47:19.060970 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:19.060972 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:19.061378 (+ 406us) server_negotiation.cc:407] Connection header received
0806 22:47:19.061485 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:19.061486 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:19.061498 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:19.061529 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:19.063034 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:19.063808 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:19.064965 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:19.065283 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:19.065326 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:19.067091 (+ 1765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:19.067092 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:19.067092 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:19.067099 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:19.069697 (+ 2598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:19.070042 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:19.070043 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:19.070043 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:19.070107 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:19.070324 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:19.070325 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:19.070326 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:19.070443 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:19.070546 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:19.070652 (+ 106us) server_negotiation.cc:299] Negotiation successful
0806 22:47:19.070726 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":24}
I20260806 22:47:20.035198 16707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:20.034855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:38077 (local address 127.6.40.66:39733)
0806 22:47:20.035022 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:20.035112 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:38077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":96,"threads_started":1}
W20260806 22:47:20.035574 16436 heartbeater.cc:644] Failed to heartbeat to 127.6.40.126:38077 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.6.40.126:38077: connect: Connection refused (error 111)
I20260806 22:47:20.036090 16708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:20.035794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:38077 (local address 127.6.40.65:48467)
0806 22:47:20.035955 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:20.036040 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:38077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":61,"threads_started":1}
W20260806 22:47:20.036337 16248 heartbeater.cc:644] Failed to heartbeat to 127.6.40.126:38077 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.6.40.126:38077: connect: Connection refused (error 111)
I20260806 22:47:20.036451 16710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:20.036177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:38077 (local address 127.6.40.67:56947)
0806 22:47:20.036315 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:20.036403 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:38077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":53,"threads_started":1}
W20260806 22:47:20.036700 16620 heartbeater.cc:644] Failed to heartbeat to 127.6.40.126:38077 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.6.40.126:38077: connect: Connection refused (error 111)
I20260806 22:47:20.539901 16711 raft_consensus.cc:482] T 00000000000000000000000000000000 P cca9c82ca527425abfcddb8d93dd4b64 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader eb7a8590eeb54daab448b3b04d3dfacf)
I20260806 22:47:20.540118 16711 raft_consensus.cc:504] T 00000000000000000000000000000000 P cca9c82ca527425abfcddb8d93dd4b64 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "eb7a8590eeb54daab448b3b04d3dfacf" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 38077 } } peers { permanent_uuid: "0016a76e8ad5476995c85a46a1fdcb6b" member_type: VOTER last_known_addr { host: "127.6.40.125" port: 42247 } attrs { promote: false } } peers { permanent_uuid: "cca9c82ca527425abfcddb8d93dd4b64" member_type: VOTER last_known_addr { host: "127.6.40.124" port: 42743 } attrs { promote: false } }
I20260806 22:47:20.540500 16711 leader_election.cc:288] T 00000000000000000000000000000000 P cca9c82ca527425abfcddb8d93dd4b64 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers eb7a8590eeb54daab448b3b04d3dfacf (127.6.40.126:38077), 0016a76e8ad5476995c85a46a1fdcb6b (127.6.40.125:42247)
I20260806 22:47:20.540822 16713 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0016a76e8ad5476995c85a46a1fdcb6b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader eb7a8590eeb54daab448b3b04d3dfacf)
I20260806 22:47:20.540956 16712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:20.540671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:38077 (local address 127.0.0.1:48564)
0806 22:47:20.540820 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:20.540905 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:38077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":63,"threads_started":1}
I20260806 22:47:20.541091 16713 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0016a76e8ad5476995c85a46a1fdcb6b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "eb7a8590eeb54daab448b3b04d3dfacf" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 38077 } } peers { permanent_uuid: "0016a76e8ad5476995c85a46a1fdcb6b" member_type: VOTER last_known_addr { host: "127.6.40.125" port: 42247 } attrs { promote: false } } peers { permanent_uuid: "cca9c82ca527425abfcddb8d93dd4b64" member_type: VOTER last_known_addr { host: "127.6.40.124" port: 42743 } attrs { promote: false } }
W20260806 22:47:20.541188 15981 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.40.126:38077: connect: Connection refused (error 111)
I20260806 22:47:20.541641 16712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:20.541522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:38077 (local address 127.0.0.1:48590)
0806 22:47:20.541580 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:20.541607 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:38077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260806 22:47:20.541745 16718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:20.541527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:38077 (local address 127.0.0.1:48578)
0806 22:47:20.541637 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:20.541706 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:38077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":57,"threads_started":1}
W20260806 22:47:20.541850 15981 leader_election.cc:334] T 00000000000000000000000000000000 P cca9c82ca527425abfcddb8d93dd4b64 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer eb7a8590eeb54daab448b3b04d3dfacf (127.6.40.126:38077): Network error: Client connection negotiation failed: client connection to 127.6.40.126:38077: connect: Connection refused (error 111)
W20260806 22:47:20.541963 15727 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.40.126:38077: connect: Connection refused (error 111)
I20260806 22:47:20.542062 16713 leader_election.cc:288] T 00000000000000000000000000000000 P 0016a76e8ad5476995c85a46a1fdcb6b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers eb7a8590eeb54daab448b3b04d3dfacf (127.6.40.126:38077), cca9c82ca527425abfcddb8d93dd4b64 (127.6.40.124:42743)
I20260806 22:47:20.542510 16718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:20.542382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:38077 (local address 127.0.0.1:48606)
0806 22:47:20.542452 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:20.542484 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:38077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260806 22:47:20.543133 15727 leader_election.cc:334] T 00000000000000000000000000000000 P 0016a76e8ad5476995c85a46a1fdcb6b [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer eb7a8590eeb54daab448b3b04d3dfacf (127.6.40.126:38077): Network error: Client connection negotiation failed: client connection to 127.6.40.126:38077: connect: Connection refused (error 111)
I20260806 22:47:20.544449 16714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:20.540727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.125:42247 (local address 127.0.0.1:52658)
0806 22:47:20.540844 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:20.540874 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:20.540938 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:20.541455 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:20.541456 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:20.541460 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:47:20.541532 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:20.541536 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:20.542579 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:20.542580 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:20.543582 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:20.543585 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:20.544181 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:20.544182 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:20.544249 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:20.544253 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:47:20.544332 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 22:47:20.544379 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":56,"threads_started":1}
I20260806 22:47:20.544443 16715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:20.540801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52658 (local address 127.6.40.125:42247)
0806 22:47:20.540930 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:20.540932 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:20.540944 (+ 12us) server_negotiation.cc:407] Connection header received
0806 22:47:20.541032 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:20.541034 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:20.541048 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:20.541612 (+ 564us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:47:20.541658 (+ 46us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:20.542490 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:20.543680 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:20.544082 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:20.544167 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:20.544181 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:20.544340 (+ 159us) server_negotiation.cc:299] Negotiation successful
0806 22:47:20.544381 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":76,"threads_started":1}
I20260806 22:47:20.544757 16719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:20.541548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.124:42743 (local address 127.0.0.1:56800)
0806 22:47:20.541932 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:20.541943 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:20.542001 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:20.542231 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:20.542232 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:20.542234 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:47:20.542283 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:20.542285 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:20.543073 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:20.543074 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:20.543923 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:20.543926 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:20.544483 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:20.544484 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:20.544556 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:20.544560 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:47:20.544641 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 22:47:20.544688 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":50,"threads_started":1}
I20260806 22:47:20.544768 16721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:20.541740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56800 (local address 127.6.40.124:42743)
0806 22:47:20.542056 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:20.542058 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:20.542066 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:47:20.542089 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:20.542090 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:20.542101 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:20.542161 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:47:20.542371 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:20.542971 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:20.544029 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:20.544388 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:20.544476 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:20.544488 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:20.544650 (+ 162us) server_negotiation.cc:299] Negotiation successful
0806 22:47:20.544689 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":75,"threads_started":1}
I20260806 22:47:20.544860 15757 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cca9c82ca527425abfcddb8d93dd4b64" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "0016a76e8ad5476995c85a46a1fdcb6b" is_pre_election: true
I20260806 22:47:20.545008 15757 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0016a76e8ad5476995c85a46a1fdcb6b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cca9c82ca527425abfcddb8d93dd4b64 in term 1.
I20260806 22:47:20.545127 16012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0016a76e8ad5476995c85a46a1fdcb6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "cca9c82ca527425abfcddb8d93dd4b64" is_pre_election: true
I20260806 22:47:20.545250 16012 raft_consensus.cc:2454] T 00000000000000000000000000000000 P cca9c82ca527425abfcddb8d93dd4b64 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0016a76e8ad5476995c85a46a1fdcb6b in term 1.
I20260806 22:47:20.545279 15980 leader_election.cc:302] T 00000000000000000000000000000000 P cca9c82ca527425abfcddb8d93dd4b64 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0016a76e8ad5476995c85a46a1fdcb6b, cca9c82ca527425abfcddb8d93dd4b64; no voters: eb7a8590eeb54daab448b3b04d3dfacf
I20260806 22:47:20.545436 16711 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cca9c82ca527425abfcddb8d93dd4b64 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260806 22:47:20.545437 15725 leader_election.cc:302] T 00000000000000000000000000000000 P 0016a76e8ad5476995c85a46a1fdcb6b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0016a76e8ad5476995c85a46a1fdcb6b, cca9c82ca527425abfcddb8d93dd4b64; no voters: eb7a8590eeb54daab448b3b04d3dfacf
I20260806 22:47:20.545485 16711 raft_consensus.cc:482] T 00000000000000000000000000000000 P cca9c82ca527425abfcddb8d93dd4b64 [term 1 FOLLOWER]: Starting leader election (detected failure of leader eb7a8590eeb54daab448b3b04d3dfacf)
I20260806 22:47:20.545511 16711 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cca9c82ca527425abfcddb8d93dd4b64 [term 1 FOLLOWER]: Advancing to term 2
I20260806 22:47:20.545555 16713 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0016a76e8ad5476995c85a46a1fdcb6b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260806 22:47:20.545593 16713 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0016a76e8ad5476995c85a46a1fdcb6b [term 1 FOLLOWER]: Starting leader election (detected failure of leader eb7a8590eeb54daab448b3b04d3dfacf)
I20260806 22:47:20.545615 16713 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0016a76e8ad5476995c85a46a1fdcb6b [term 1 FOLLOWER]: Advancing to term 2
I20260806 22:47:20.546504 16711 raft_consensus.cc:504] T 00000000000000000000000000000000 P cca9c82ca527425abfcddb8d93dd4b64 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "eb7a8590eeb54daab448b3b04d3dfacf" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 38077 } } peers { permanent_uuid: "0016a76e8ad5476995c85a46a1fdcb6b" member_type: VOTER last_known_addr { host: "127.6.40.125" port: 42247 } attrs { promote: false } } peers { permanent_uuid: "cca9c82ca527425abfcddb8d93dd4b64" member_type: VOTER last_known_addr { host: "127.6.40.124" port: 42743 } attrs { promote: false } }
I20260806 22:47:20.546504 16713 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0016a76e8ad5476995c85a46a1fdcb6b [term 2 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "eb7a8590eeb54daab448b3b04d3dfacf" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 38077 } } peers { permanent_uuid: "0016a76e8ad5476995c85a46a1fdcb6b" member_type: VOTER last_known_addr { host: "127.6.40.125" port: 42247 } attrs { promote: false } } peers { permanent_uuid: "cca9c82ca527425abfcddb8d93dd4b64" member_type: VOTER last_known_addr { host: "127.6.40.124" port: 42743 } attrs { promote: false } }
I20260806 22:47:20.546635 16713 leader_election.cc:288] T 00000000000000000000000000000000 P 0016a76e8ad5476995c85a46a1fdcb6b [CANDIDATE]: Term 2 election: Requested vote from peers eb7a8590eeb54daab448b3b04d3dfacf (127.6.40.126:38077), cca9c82ca527425abfcddb8d93dd4b64 (127.6.40.124:42743)
I20260806 22:47:20.546635 16711 leader_election.cc:288] T 00000000000000000000000000000000 P cca9c82ca527425abfcddb8d93dd4b64 [CANDIDATE]: Term 2 election: Requested vote from peers eb7a8590eeb54daab448b3b04d3dfacf (127.6.40.126:38077), 0016a76e8ad5476995c85a46a1fdcb6b (127.6.40.125:42247)
I20260806 22:47:20.546849 16714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:20.546752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:38077 (local address 127.0.0.1:48614)
0806 22:47:20.546791 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:20.546822 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:38077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260806 22:47:20.546912 15757 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cca9c82ca527425abfcddb8d93dd4b64" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "0016a76e8ad5476995c85a46a1fdcb6b"
I20260806 22:47:20.546945 16012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0016a76e8ad5476995c85a46a1fdcb6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "cca9c82ca527425abfcddb8d93dd4b64"
I20260806 22:47:20.547017 16719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:20.546752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:38077 (local address 127.0.0.1:48612)
0806 22:47:20.546964 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:20.546994 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:38077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193}
I20260806 22:47:20.547031 16012 raft_consensus.cc:2379] T 00000000000000000000000000000000 P cca9c82ca527425abfcddb8d93dd4b64 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 0016a76e8ad5476995c85a46a1fdcb6b in current term 2: Already voted for candidate cca9c82ca527425abfcddb8d93dd4b64 in this term.
I20260806 22:47:20.547030 15757 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 0016a76e8ad5476995c85a46a1fdcb6b [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate cca9c82ca527425abfcddb8d93dd4b64 in current term 2: Already voted for candidate 0016a76e8ad5476995c85a46a1fdcb6b in this term.
I20260806 22:47:20.547390 16719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:20.547233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:38077 (local address 127.0.0.1:48636)
0806 22:47:20.547296 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:20.547322 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:38077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260806 22:47:20.547434 16714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:20.547349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:38077 (local address 127.0.0.1:48644)
0806 22:47:20.547380 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:20.547411 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:38077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260806 22:47:20.547497 15727 leader_election.cc:334] T 00000000000000000000000000000000 P 0016a76e8ad5476995c85a46a1fdcb6b [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer eb7a8590eeb54daab448b3b04d3dfacf (127.6.40.126:38077): Network error: Client connection negotiation failed: client connection to 127.6.40.126:38077: connect: Connection refused (error 111)
I20260806 22:47:20.547554 15727 leader_election.cc:302] T 00000000000000000000000000000000 P 0016a76e8ad5476995c85a46a1fdcb6b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0016a76e8ad5476995c85a46a1fdcb6b; no voters: cca9c82ca527425abfcddb8d93dd4b64, eb7a8590eeb54daab448b3b04d3dfacf
I20260806 22:47:20.547614 16713 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 0016a76e8ad5476995c85a46a1fdcb6b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260806 22:47:20.547663 15981 leader_election.cc:334] T 00000000000000000000000000000000 P cca9c82ca527425abfcddb8d93dd4b64 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer eb7a8590eeb54daab448b3b04d3dfacf (127.6.40.126:38077): Network error: Client connection negotiation failed: client connection to 127.6.40.126:38077: connect: Connection refused (error 111)
I20260806 22:47:20.547708 15981 leader_election.cc:302] T 00000000000000000000000000000000 P cca9c82ca527425abfcddb8d93dd4b64 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cca9c82ca527425abfcddb8d93dd4b64; no voters: 0016a76e8ad5476995c85a46a1fdcb6b, eb7a8590eeb54daab448b3b04d3dfacf
I20260806 22:47:20.547791 16711 raft_consensus.cc:2726] T 00000000000000000000000000000000 P cca9c82ca527425abfcddb8d93dd4b64 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260806 22:47:21.036304 16724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:21.035973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:38077 (local address 127.6.40.66:45303)
0806 22:47:21.036164 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:21.036217 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:38077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":120,"threads_started":1}
I20260806 22:47:21.037283 16729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:21.037061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:38077 (local address 127.6.40.67:45747)
0806 22:47:21.037182 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:21.037229 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:38077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":64,"threads_started":1}
I20260806 22:47:21.037626 16728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:21.036808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:38077 (local address 127.6.40.65:43763)
0806 22:47:21.037531 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:21.037574 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:38077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":92,"threads_started":1}
I20260806 22:47:22.038050 16731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:22.037705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:38077 (local address 127.6.40.67:50005)
0806 22:47:22.037841 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:22.037989 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:38077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":63,"threads_started":1}
I20260806 22:47:22.037926 16730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:22.037154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:38077 (local address 127.6.40.66:46437)
0806 22:47:22.037804 (+ 650us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:22.037857 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:38077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":76,"threads_started":1}
I20260806 22:47:22.038362 16732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:22.038051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:38077 (local address 127.6.40.65:34385)
0806 22:47:22.038274 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:22.038314 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:38077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":54,"threads_started":1}
I20260806 22:47:22.254588 16733 raft_consensus.cc:482] T 00000000000000000000000000000000 P cca9c82ca527425abfcddb8d93dd4b64 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:47:22.254698 16733 raft_consensus.cc:504] T 00000000000000000000000000000000 P cca9c82ca527425abfcddb8d93dd4b64 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "eb7a8590eeb54daab448b3b04d3dfacf" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 38077 } } peers { permanent_uuid: "0016a76e8ad5476995c85a46a1fdcb6b" member_type: VOTER last_known_addr { host: "127.6.40.125" port: 42247 } attrs { promote: false } } peers { permanent_uuid: "cca9c82ca527425abfcddb8d93dd4b64" member_type: VOTER last_known_addr { host: "127.6.40.124" port: 42743 } attrs { promote: false } }
I20260806 22:47:22.254938 16733 leader_election.cc:288] T 00000000000000000000000000000000 P cca9c82ca527425abfcddb8d93dd4b64 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers eb7a8590eeb54daab448b3b04d3dfacf (127.6.40.126:38077), 0016a76e8ad5476995c85a46a1fdcb6b (127.6.40.125:42247)
I20260806 22:47:22.255263 15757 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cca9c82ca527425abfcddb8d93dd4b64" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "0016a76e8ad5476995c85a46a1fdcb6b" is_pre_election: true
I20260806 22:47:22.255363 16734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:22.255121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:38077 (local address 127.0.0.1:48876)
0806 22:47:22.255251 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:22.255288 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:38077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":78,"threads_started":1}
I20260806 22:47:22.255421 15757 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0016a76e8ad5476995c85a46a1fdcb6b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cca9c82ca527425abfcddb8d93dd4b64 in term 2.
I20260806 22:47:22.255628 15980 leader_election.cc:302] T 00000000000000000000000000000000 P cca9c82ca527425abfcddb8d93dd4b64 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0016a76e8ad5476995c85a46a1fdcb6b, cca9c82ca527425abfcddb8d93dd4b64; no voters:
I20260806 22:47:22.255741 16733 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cca9c82ca527425abfcddb8d93dd4b64 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260806 22:47:22.255782 16733 raft_consensus.cc:482] T 00000000000000000000000000000000 P cca9c82ca527425abfcddb8d93dd4b64 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:47:22.255803 16733 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cca9c82ca527425abfcddb8d93dd4b64 [term 2 FOLLOWER]: Advancing to term 3
I20260806 22:47:22.255952 16734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:22.255784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:38077 (local address 127.0.0.1:48890)
0806 22:47:22.255889 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:22.255920 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:38077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
W20260806 22:47:22.256042 15981 leader_election.cc:334] T 00000000000000000000000000000000 P cca9c82ca527425abfcddb8d93dd4b64 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer eb7a8590eeb54daab448b3b04d3dfacf (127.6.40.126:38077): Network error: Client connection negotiation failed: client connection to 127.6.40.126:38077: connect: Connection refused (error 111)
I20260806 22:47:22.256670 16733 raft_consensus.cc:504] T 00000000000000000000000000000000 P cca9c82ca527425abfcddb8d93dd4b64 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "eb7a8590eeb54daab448b3b04d3dfacf" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 38077 } } peers { permanent_uuid: "0016a76e8ad5476995c85a46a1fdcb6b" member_type: VOTER last_known_addr { host: "127.6.40.125" port: 42247 } attrs { promote: false } } peers { permanent_uuid: "cca9c82ca527425abfcddb8d93dd4b64" member_type: VOTER last_known_addr { host: "127.6.40.124" port: 42743 } attrs { promote: false } }
I20260806 22:47:22.256808 16733 leader_election.cc:288] T 00000000000000000000000000000000 P cca9c82ca527425abfcddb8d93dd4b64 [CANDIDATE]: Term 3 election: Requested vote from peers eb7a8590eeb54daab448b3b04d3dfacf (127.6.40.126:38077), 0016a76e8ad5476995c85a46a1fdcb6b (127.6.40.125:42247)
I20260806 22:47:22.257030 16734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:22.256943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:38077 (local address 127.0.0.1:48894)
0806 22:47:22.256978 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:22.257006 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:38077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260806 22:47:22.257018 15757 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cca9c82ca527425abfcddb8d93dd4b64" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "0016a76e8ad5476995c85a46a1fdcb6b"
I20260806 22:47:22.257129 15757 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0016a76e8ad5476995c85a46a1fdcb6b [term 2 FOLLOWER]: Advancing to term 3
I20260806 22:47:22.257397 16734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:22.257295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:38077 (local address 127.0.0.1:48908)
0806 22:47:22.257349 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:22.257378 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:38077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260806 22:47:22.257485 15981 leader_election.cc:334] T 00000000000000000000000000000000 P cca9c82ca527425abfcddb8d93dd4b64 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer eb7a8590eeb54daab448b3b04d3dfacf (127.6.40.126:38077): Network error: Client connection negotiation failed: client connection to 127.6.40.126:38077: connect: Connection refused (error 111)
I20260806 22:47:22.257997 15757 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0016a76e8ad5476995c85a46a1fdcb6b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cca9c82ca527425abfcddb8d93dd4b64 in term 3.
I20260806 22:47:22.258193 15980 leader_election.cc:302] T 00000000000000000000000000000000 P cca9c82ca527425abfcddb8d93dd4b64 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0016a76e8ad5476995c85a46a1fdcb6b, cca9c82ca527425abfcddb8d93dd4b64; no voters: eb7a8590eeb54daab448b3b04d3dfacf
I20260806 22:47:22.258308 16733 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cca9c82ca527425abfcddb8d93dd4b64 [term 3 FOLLOWER]: Leader election won for term 3
I20260806 22:47:22.258461 16733 raft_consensus.cc:686] T 00000000000000000000000000000000 P cca9c82ca527425abfcddb8d93dd4b64 [term 3 LEADER]: Becoming Leader. State: Replica: cca9c82ca527425abfcddb8d93dd4b64, State: Running, Role: LEADER
I20260806 22:47:22.258559 16733 consensus_queue.cc:237] T 00000000000000000000000000000000 P cca9c82ca527425abfcddb8d93dd4b64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "eb7a8590eeb54daab448b3b04d3dfacf" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 38077 } } peers { permanent_uuid: "0016a76e8ad5476995c85a46a1fdcb6b" member_type: VOTER last_known_addr { host: "127.6.40.125" port: 42247 } attrs { promote: false } } peers { permanent_uuid: "cca9c82ca527425abfcddb8d93dd4b64" member_type: VOTER last_known_addr { host: "127.6.40.124" port: 42743 } attrs { promote: false } }
I20260806 22:47:22.259007 16736 sys_catalog.cc:455] T 00000000000000000000000000000000 P cca9c82ca527425abfcddb8d93dd4b64 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cca9c82ca527425abfcddb8d93dd4b64. Latest consensus state: current_term: 3 leader_uuid: "cca9c82ca527425abfcddb8d93dd4b64" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "eb7a8590eeb54daab448b3b04d3dfacf" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 38077 } } peers { permanent_uuid: "0016a76e8ad5476995c85a46a1fdcb6b" member_type: VOTER last_known_addr { host: "127.6.40.125" port: 42247 } attrs { promote: false } } peers { permanent_uuid: "cca9c82ca527425abfcddb8d93dd4b64" member_type: VOTER last_known_addr { host: "127.6.40.124" port: 42743 } attrs { promote: false } } }
I20260806 22:47:22.259115 16736 sys_catalog.cc:458] T 00000000000000000000000000000000 P cca9c82ca527425abfcddb8d93dd4b64 [sys.catalog]: This master's current role is: LEADER
I20260806 22:47:22.259395 16738 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:47:22.259891 16738 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:47:22.259965 16738 catalog_manager.cc:1259] Loaded cluster ID: 16746f209dcb48eb906c4d27294225e8
I20260806 22:47:22.259997 16738 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:47:22.260167 16738 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:47:22.260282 16738 catalog_manager.cc:5879] T 00000000000000000000000000000000 P cca9c82ca527425abfcddb8d93dd4b64: Loaded TSK: 0
I20260806 22:47:22.260504 16738 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:47:22.316010 15986 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56788:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:47:22.317919 15986 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:47:22.319181 16734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:22.319029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:38077 (local address 127.0.0.1:48916)
0806 22:47:22.319108 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:22.319149 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:38077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260806 22:47:22.319181 15757 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0016a76e8ad5476995c85a46a1fdcb6b [term 3 FOLLOWER]: Refusing update from remote peer cca9c82ca527425abfcddb8d93dd4b64: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260806 22:47:22.319588 16734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:22.319445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:38077 (local address 127.0.0.1:48920)
0806 22:47:22.319506 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:22.319541 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:38077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260806 22:47:22.319521 16733 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cca9c82ca527425abfcddb8d93dd4b64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0016a76e8ad5476995c85a46a1fdcb6b" member_type: VOTER last_known_addr { host: "127.6.40.125" port: 42247 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260806 22:47:22.319682 15981 consensus_peers.cc:487] T 00000000000000000000000000000000 P cca9c82ca527425abfcddb8d93dd4b64 -> Peer eb7a8590eeb54daab448b3b04d3dfacf (127.6.40.126:38077): Couldn't send request to peer eb7a8590eeb54daab448b3b04d3dfacf. Status: Network error: Client connection negotiation failed: client connection to 127.6.40.126:38077: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 22:47:22.320268 16741 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0016a76e8ad5476995c85a46a1fdcb6b [sys.catalog]: SysCatalogTable state changed. Reason: New leader cca9c82ca527425abfcddb8d93dd4b64. Latest consensus state: current_term: 3 leader_uuid: "cca9c82ca527425abfcddb8d93dd4b64" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "eb7a8590eeb54daab448b3b04d3dfacf" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 38077 } } peers { permanent_uuid: "0016a76e8ad5476995c85a46a1fdcb6b" member_type: VOTER last_known_addr { host: "127.6.40.125" port: 42247 } attrs { promote: false } } peers { permanent_uuid: "cca9c82ca527425abfcddb8d93dd4b64" member_type: VOTER last_known_addr { host: "127.6.40.124" port: 42743 } attrs { promote: false } } }
I20260806 22:47:22.320441 16741 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0016a76e8ad5476995c85a46a1fdcb6b [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:47:22.320968 16733 sys_catalog.cc:455] T 00000000000000000000000000000000 P cca9c82ca527425abfcddb8d93dd4b64 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "cca9c82ca527425abfcddb8d93dd4b64" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "eb7a8590eeb54daab448b3b04d3dfacf" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 38077 } } peers { permanent_uuid: "0016a76e8ad5476995c85a46a1fdcb6b" member_type: VOTER last_known_addr { host: "127.6.40.125" port: 42247 } attrs { promote: false } } peers { permanent_uuid: "cca9c82ca527425abfcddb8d93dd4b64" member_type: VOTER last_known_addr { host: "127.6.40.124" port: 42743 } attrs { promote: false } } }
I20260806 22:47:22.321033 16742 mvcc.cc:205] Tried to move back new op lower bound from 7315687187736682496 to 7315687187491905536. Current Snapshot: MvccSnapshot[applied={T|T < 7315687187736682496}]
I20260806 22:47:22.321183 16741 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0016a76e8ad5476995c85a46a1fdcb6b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "cca9c82ca527425abfcddb8d93dd4b64" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "eb7a8590eeb54daab448b3b04d3dfacf" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 38077 } } peers { permanent_uuid: "0016a76e8ad5476995c85a46a1fdcb6b" member_type: VOTER last_known_addr { host: "127.6.40.125" port: 42247 } attrs { promote: false } } peers { permanent_uuid: "cca9c82ca527425abfcddb8d93dd4b64" member_type: VOTER last_known_addr { host: "127.6.40.124" port: 42743 } attrs { promote: false } } }
I20260806 22:47:22.321267 16733 sys_catalog.cc:458] T 00000000000000000000000000000000 P cca9c82ca527425abfcddb8d93dd4b64 [sys.catalog]: This master's current role is: LEADER
I20260806 22:47:22.321367 16741 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0016a76e8ad5476995c85a46a1fdcb6b [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:47:22.326953 16734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:22.323114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.66:34829 (local address 127.0.0.1:47980)
0806 22:47:22.323155 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:22.323165 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:22.323208 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:22.323711 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:22.323712 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:22.323714 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:47:22.323775 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:22.323778 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:22.324518 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:22.324519 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:22.325674 (+ 1155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:22.325677 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:22.326659 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:22.326660 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:22.326727 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:22.326730 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:47:22.326790 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 22:47:22.326838 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260806 22:47:22.327005 16749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:22.323189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47980 (local address 127.6.40.66:34829)
0806 22:47:22.323427 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:22.323430 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:22.323441 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:47:22.323474 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:22.323475 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:22.323488 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:22.323572 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:47:22.323874 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:22.324432 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:22.326055 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:22.326572 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:22.326658 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:22.326673 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:22.326796 (+ 123us) server_negotiation.cc:299] Negotiation successful
0806 22:47:22.326868 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":214,"thread_start_us":66,"threads_started":1}
I20260806 22:47:22.326982 16750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:22.323404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.65:43189 (local address 127.0.0.1:47214)
0806 22:47:22.323505 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:22.323516 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:22.323557 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:22.323890 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:22.323891 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:22.323893 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:47:22.323930 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:22.323931 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:22.324863 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:22.324864 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:22.325867 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:22.325870 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:22.326668 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:22.326669 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:22.326753 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:22.326758 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:47:22.326793 (+ 35us) client_negotiation.cc:240] Negotiation successful
0806 22:47:22.326864 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":85,"mutex_wait_us":41,"thread_start_us":69,"threads_started":1}
I20260806 22:47:22.327208 16748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:22.323164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.67:45455 (local address 127.0.0.1:58324)
0806 22:47:22.323300 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:22.323309 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:22.323372 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:22.323841 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:22.323841 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:22.323844 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:47:22.323894 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:22.323896 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:22.324830 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:22.324831 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:22.325993 (+ 1162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:22.325995 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:22.326887 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:22.326887 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:22.326950 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:22.326953 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:47:22.326989 (+ 36us) client_negotiation.cc:240] Negotiation successful
0806 22:47:22.327033 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":82,"threads_started":1}
I20260806 22:47:22.327422 16752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:22.323594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47214 (local address 127.6.40.65:43189)
0806 22:47:22.323720 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:22.323723 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:22.323731 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:47:22.323761 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:22.323761 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:22.323773 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:22.323873 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:47:22.323987 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:22.324733 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:22.325979 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:22.326567 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:22.327199 (+ 632us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:22.327216 (+ 17us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:22.327237 (+ 21us) server_negotiation.cc:299] Negotiation successful
0806 22:47:22.327348 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":74,"threads_started":1}
I20260806 22:47:22.327500 16751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:22.323398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58324 (local address 127.6.40.67:45455)
0806 22:47:22.323671 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:22.323673 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:22.323683 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:47:22.323714 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:22.323715 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:22.323727 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:22.323961 (+ 234us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:47:22.324006 (+ 45us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:22.324750 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:22.326152 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:22.326788 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:22.327320 (+ 532us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:22.327348 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:22.327369 (+ 21us) server_negotiation.cc:299] Negotiation successful
0806 22:47:22.327415 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":249,"thread_start_us":80,"threads_started":1}
I20260806 22:47:22.327667 16158 tablet_service.cc:1467] Processing CreateTablet for tablet f28ae5f6eeac4a6b8200bf1976754289 (DEFAULT_TABLE table=test-table [id=b9f560526246486dbd27142a233f4f77]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:47:22.327826 16158 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f28ae5f6eeac4a6b8200bf1976754289. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:47:22.327914 16338 tablet_service.cc:1467] Processing CreateTablet for tablet f28ae5f6eeac4a6b8200bf1976754289 (DEFAULT_TABLE table=test-table [id=b9f560526246486dbd27142a233f4f77]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:47:22.328068 16338 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f28ae5f6eeac4a6b8200bf1976754289. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:47:22.329557 16753 tablet_bootstrap.cc:492] T f28ae5f6eeac4a6b8200bf1976754289 P 35a7f06767bb4b81999ee9fa87d12aa3: Bootstrap starting.
I20260806 22:47:22.329588 16532 tablet_service.cc:1467] Processing CreateTablet for tablet f28ae5f6eeac4a6b8200bf1976754289 (DEFAULT_TABLE table=test-table [id=b9f560526246486dbd27142a233f4f77]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:47:22.329717 16532 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f28ae5f6eeac4a6b8200bf1976754289. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:47:22.329939 16753 tablet_bootstrap.cc:654] T f28ae5f6eeac4a6b8200bf1976754289 P 35a7f06767bb4b81999ee9fa87d12aa3: Neither blocks nor log segments found. Creating new log.
I20260806 22:47:22.330611 16753 tablet_bootstrap.cc:492] T f28ae5f6eeac4a6b8200bf1976754289 P 35a7f06767bb4b81999ee9fa87d12aa3: No bootstrap required, opened a new log
I20260806 22:47:22.330655 16753 ts_tablet_manager.cc:1397] T f28ae5f6eeac4a6b8200bf1976754289 P 35a7f06767bb4b81999ee9fa87d12aa3: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 22:47:22.330777 16753 raft_consensus.cc:348] T f28ae5f6eeac4a6b8200bf1976754289 P 35a7f06767bb4b81999ee9fa87d12aa3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1fec59295474125ae87691d06eaf345" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 34829 } } peers { permanent_uuid: "62e43f48d5464af9bcbd68d8fb9877d4" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45455 } } peers { permanent_uuid: "35a7f06767bb4b81999ee9fa87d12aa3" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 43189 } }
I20260806 22:47:22.330830 16753 raft_consensus.cc:374] T f28ae5f6eeac4a6b8200bf1976754289 P 35a7f06767bb4b81999ee9fa87d12aa3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:47:22.330852 16753 raft_consensus.cc:729] T f28ae5f6eeac4a6b8200bf1976754289 P 35a7f06767bb4b81999ee9fa87d12aa3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35a7f06767bb4b81999ee9fa87d12aa3, State: Initialized, Role: FOLLOWER
I20260806 22:47:22.330910 16753 consensus_queue.cc:260] T f28ae5f6eeac4a6b8200bf1976754289 P 35a7f06767bb4b81999ee9fa87d12aa3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1fec59295474125ae87691d06eaf345" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 34829 } } peers { permanent_uuid: "62e43f48d5464af9bcbd68d8fb9877d4" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45455 } } peers { permanent_uuid: "35a7f06767bb4b81999ee9fa87d12aa3" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 43189 } }
I20260806 22:47:22.331058 16757 tablet_bootstrap.cc:492] T f28ae5f6eeac4a6b8200bf1976754289 P 62e43f48d5464af9bcbd68d8fb9877d4: Bootstrap starting.
I20260806 22:47:22.331125 16753 ts_tablet_manager.cc:1428] T f28ae5f6eeac4a6b8200bf1976754289 P 35a7f06767bb4b81999ee9fa87d12aa3: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 22:47:22.331384 16755 tablet_bootstrap.cc:492] T f28ae5f6eeac4a6b8200bf1976754289 P d1fec59295474125ae87691d06eaf345: Bootstrap starting.
I20260806 22:47:22.331463 16732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:22.331350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:38077 (local address 127.6.40.65:46013)
0806 22:47:22.331399 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:22.331428 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:38077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260806 22:47:22.331499 16757 tablet_bootstrap.cc:654] T f28ae5f6eeac4a6b8200bf1976754289 P 62e43f48d5464af9bcbd68d8fb9877d4: Neither blocks nor log segments found. Creating new log.
I20260806 22:47:22.331897 16755 tablet_bootstrap.cc:654] T f28ae5f6eeac4a6b8200bf1976754289 P d1fec59295474125ae87691d06eaf345: Neither blocks nor log segments found. Creating new log.
I20260806 22:47:22.332228 16247 heartbeater.cc:503] Master 127.6.40.124:42743 was elected leader, sending a full tablet report...
I20260806 22:47:22.332415 16732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:22.332317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:38077 (local address 127.6.40.65:52549)
0806 22:47:22.332358 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:22.332384 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:38077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260806 22:47:22.332546 16755 tablet_bootstrap.cc:492] T f28ae5f6eeac4a6b8200bf1976754289 P d1fec59295474125ae87691d06eaf345: No bootstrap required, opened a new log
I20260806 22:47:22.332602 16755 ts_tablet_manager.cc:1397] T f28ae5f6eeac4a6b8200bf1976754289 P d1fec59295474125ae87691d06eaf345: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 22:47:22.332743 16755 raft_consensus.cc:348] T f28ae5f6eeac4a6b8200bf1976754289 P d1fec59295474125ae87691d06eaf345 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1fec59295474125ae87691d06eaf345" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 34829 } } peers { permanent_uuid: "62e43f48d5464af9bcbd68d8fb9877d4" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45455 } } peers { permanent_uuid: "35a7f06767bb4b81999ee9fa87d12aa3" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 43189 } }
I20260806 22:47:22.332827 16755 raft_consensus.cc:374] T f28ae5f6eeac4a6b8200bf1976754289 P d1fec59295474125ae87691d06eaf345 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:47:22.332885 16755 raft_consensus.cc:729] T f28ae5f6eeac4a6b8200bf1976754289 P d1fec59295474125ae87691d06eaf345 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1fec59295474125ae87691d06eaf345, State: Initialized, Role: FOLLOWER
I20260806 22:47:22.332906 16757 tablet_bootstrap.cc:492] T f28ae5f6eeac4a6b8200bf1976754289 P 62e43f48d5464af9bcbd68d8fb9877d4: No bootstrap required, opened a new log
I20260806 22:47:22.332958 16757 ts_tablet_manager.cc:1397] T f28ae5f6eeac4a6b8200bf1976754289 P 62e43f48d5464af9bcbd68d8fb9877d4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:47:22.332932 16755 consensus_queue.cc:260] T f28ae5f6eeac4a6b8200bf1976754289 P d1fec59295474125ae87691d06eaf345 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1fec59295474125ae87691d06eaf345" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 34829 } } peers { permanent_uuid: "62e43f48d5464af9bcbd68d8fb9877d4" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45455 } } peers { permanent_uuid: "35a7f06767bb4b81999ee9fa87d12aa3" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 43189 } }
I20260806 22:47:22.333156 16755 ts_tablet_manager.cc:1428] T f28ae5f6eeac4a6b8200bf1976754289 P d1fec59295474125ae87691d06eaf345: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260806 22:47:22.333132 16757 raft_consensus.cc:348] T f28ae5f6eeac4a6b8200bf1976754289 P 62e43f48d5464af9bcbd68d8fb9877d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1fec59295474125ae87691d06eaf345" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 34829 } } peers { permanent_uuid: "62e43f48d5464af9bcbd68d8fb9877d4" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45455 } } peers { permanent_uuid: "35a7f06767bb4b81999ee9fa87d12aa3" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 43189 } }
I20260806 22:47:22.333210 16757 raft_consensus.cc:374] T f28ae5f6eeac4a6b8200bf1976754289 P 62e43f48d5464af9bcbd68d8fb9877d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:47:22.333235 16757 raft_consensus.cc:729] T f28ae5f6eeac4a6b8200bf1976754289 P 62e43f48d5464af9bcbd68d8fb9877d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62e43f48d5464af9bcbd68d8fb9877d4, State: Initialized, Role: FOLLOWER
I20260806 22:47:22.333289 16757 consensus_queue.cc:260] T f28ae5f6eeac4a6b8200bf1976754289 P 62e43f48d5464af9bcbd68d8fb9877d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1fec59295474125ae87691d06eaf345" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 34829 } } peers { permanent_uuid: "62e43f48d5464af9bcbd68d8fb9877d4" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45455 } } peers { permanent_uuid: "35a7f06767bb4b81999ee9fa87d12aa3" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 43189 } }
I20260806 22:47:22.333549 16757 ts_tablet_manager.cc:1428] T f28ae5f6eeac4a6b8200bf1976754289 P 62e43f48d5464af9bcbd68d8fb9877d4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 22:47:22.333909 16731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:22.333807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:38077 (local address 127.6.40.67:45663)
0806 22:47:22.333851 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:22.333878 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:38077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260806 22:47:22.333925 16730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:22.333818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:38077 (local address 127.6.40.66:41431)
0806 22:47:22.333860 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:22.333890 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:38077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260806 22:47:22.334101 16435 heartbeater.cc:503] Master 127.6.40.124:42743 was elected leader, sending a full tablet report...
I20260806 22:47:22.334353 16619 heartbeater.cc:503] Master 127.6.40.124:42743 was elected leader, sending a full tablet report...
I20260806 22:47:22.334544 16731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:22.334451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:38077 (local address 127.6.40.67:54363)
0806 22:47:22.334486 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:22.334518 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:38077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
I20260806 22:47:22.438373 16763 raft_consensus.cc:482] T f28ae5f6eeac4a6b8200bf1976754289 P 62e43f48d5464af9bcbd68d8fb9877d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:47:22.438524 16763 raft_consensus.cc:504] T f28ae5f6eeac4a6b8200bf1976754289 P 62e43f48d5464af9bcbd68d8fb9877d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1fec59295474125ae87691d06eaf345" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 34829 } } peers { permanent_uuid: "62e43f48d5464af9bcbd68d8fb9877d4" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45455 } } peers { permanent_uuid: "35a7f06767bb4b81999ee9fa87d12aa3" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 43189 } }
I20260806 22:47:22.438954 16763 leader_election.cc:288] T f28ae5f6eeac4a6b8200bf1976754289 P 62e43f48d5464af9bcbd68d8fb9877d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d1fec59295474125ae87691d06eaf345 (127.6.40.66:34829), 35a7f06767bb4b81999ee9fa87d12aa3 (127.6.40.65:43189)
I20260806 22:47:22.442318 16752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:22.439401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.67:46229 (local address 127.6.40.65:43189)
0806 22:47:22.439450 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:22.439451 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:22.439587 (+ 136us) server_negotiation.cc:407] Connection header received
0806 22:47:22.439622 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:22.439623 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:22.439640 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:22.439701 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:47:22.439927 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:22.440388 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:22.441499 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:22.441930 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:22.442012 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:22.442028 (+ 16us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:22.442213 (+ 185us) server_negotiation.cc:299] Negotiation successful
0806 22:47:22.442249 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 22:47:22.442337 16765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:22.439154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.65:43189 (local address 127.6.40.67:46229)
0806 22:47:22.439503 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:22.439513 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:22.439566 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:22.439761 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:22.439763 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:22.439765 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:47:22.439834 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:22.439836 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:22.440504 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:22.440505 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:22.441382 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:22.441385 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:22.442038 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:22.442039 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:22.442108 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:22.442113 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:47:22.442175 (+ 62us) client_negotiation.cc:240] Negotiation successful
0806 22:47:22.442239 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":128,"threads_started":1}
I20260806 22:47:22.442387 16749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:22.439205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.67:50439 (local address 127.6.40.66:34829)
0806 22:47:22.439253 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:22.439255 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:22.439265 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:47:22.439365 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:22.439366 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:22.439381 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:22.439429 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:47:22.439761 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:22.440376 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:22.441470 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:22.441900 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:22.442029 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:22.442044 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:22.442261 (+ 217us) server_negotiation.cc:299] Negotiation successful
0806 22:47:22.442308 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 22:47:22.442418 16731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:22.439142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.66:34829 (local address 127.6.40.67:50439)
0806 22:47:22.439200 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:22.439212 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:22.439271 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:22.439486 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:22.439487 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:22.439489 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:47:22.439588 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:22.439590 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:22.440469 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:22.440470 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:22.441369 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:22.441371 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:22.442081 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:22.442083 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:22.442145 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:22.442149 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:47:22.442252 (+ 103us) client_negotiation.cc:240] Negotiation successful
0806 22:47:22.442302 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260806 22:47:22.442670 16183 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f28ae5f6eeac4a6b8200bf1976754289" candidate_uuid: "62e43f48d5464af9bcbd68d8fb9877d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35a7f06767bb4b81999ee9fa87d12aa3" is_pre_election: true
I20260806 22:47:22.442763 16183 raft_consensus.cc:2454] T f28ae5f6eeac4a6b8200bf1976754289 P 35a7f06767bb4b81999ee9fa87d12aa3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 62e43f48d5464af9bcbd68d8fb9877d4 in term 0.
I20260806 22:47:22.442724 16366 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f28ae5f6eeac4a6b8200bf1976754289" candidate_uuid: "62e43f48d5464af9bcbd68d8fb9877d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1fec59295474125ae87691d06eaf345" is_pre_election: true
I20260806 22:47:22.442911 16366 raft_consensus.cc:2454] T f28ae5f6eeac4a6b8200bf1976754289 P d1fec59295474125ae87691d06eaf345 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 62e43f48d5464af9bcbd68d8fb9877d4 in term 0.
I20260806 22:47:22.442996 16464 leader_election.cc:302] T f28ae5f6eeac4a6b8200bf1976754289 P 62e43f48d5464af9bcbd68d8fb9877d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35a7f06767bb4b81999ee9fa87d12aa3, 62e43f48d5464af9bcbd68d8fb9877d4; no voters:
I20260806 22:47:22.443267 16763 raft_consensus.cc:2781] T f28ae5f6eeac4a6b8200bf1976754289 P 62e43f48d5464af9bcbd68d8fb9877d4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:47:22.443336 16763 raft_consensus.cc:482] T f28ae5f6eeac4a6b8200bf1976754289 P 62e43f48d5464af9bcbd68d8fb9877d4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:47:22.443363 16763 raft_consensus.cc:3037] T f28ae5f6eeac4a6b8200bf1976754289 P 62e43f48d5464af9bcbd68d8fb9877d4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:47:22.444177 16763 raft_consensus.cc:504] T f28ae5f6eeac4a6b8200bf1976754289 P 62e43f48d5464af9bcbd68d8fb9877d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1fec59295474125ae87691d06eaf345" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 34829 } } peers { permanent_uuid: "62e43f48d5464af9bcbd68d8fb9877d4" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45455 } } peers { permanent_uuid: "35a7f06767bb4b81999ee9fa87d12aa3" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 43189 } }
I20260806 22:47:22.444351 16763 leader_election.cc:288] T f28ae5f6eeac4a6b8200bf1976754289 P 62e43f48d5464af9bcbd68d8fb9877d4 [CANDIDATE]: Term 1 election: Requested vote from peers d1fec59295474125ae87691d06eaf345 (127.6.40.66:34829), 35a7f06767bb4b81999ee9fa87d12aa3 (127.6.40.65:43189)
I20260806 22:47:22.444530 16183 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f28ae5f6eeac4a6b8200bf1976754289" candidate_uuid: "62e43f48d5464af9bcbd68d8fb9877d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35a7f06767bb4b81999ee9fa87d12aa3"
I20260806 22:47:22.444525 16366 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f28ae5f6eeac4a6b8200bf1976754289" candidate_uuid: "62e43f48d5464af9bcbd68d8fb9877d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1fec59295474125ae87691d06eaf345"
I20260806 22:47:22.444617 16183 raft_consensus.cc:3037] T f28ae5f6eeac4a6b8200bf1976754289 P 35a7f06767bb4b81999ee9fa87d12aa3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:47:22.444617 16366 raft_consensus.cc:3037] T f28ae5f6eeac4a6b8200bf1976754289 P d1fec59295474125ae87691d06eaf345 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:47:22.445526 16183 raft_consensus.cc:2454] T f28ae5f6eeac4a6b8200bf1976754289 P 35a7f06767bb4b81999ee9fa87d12aa3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62e43f48d5464af9bcbd68d8fb9877d4 in term 1.
I20260806 22:47:22.445660 16366 raft_consensus.cc:2454] T f28ae5f6eeac4a6b8200bf1976754289 P d1fec59295474125ae87691d06eaf345 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62e43f48d5464af9bcbd68d8fb9877d4 in term 1.
I20260806 22:47:22.445753 16464 leader_election.cc:302] T f28ae5f6eeac4a6b8200bf1976754289 P 62e43f48d5464af9bcbd68d8fb9877d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35a7f06767bb4b81999ee9fa87d12aa3, 62e43f48d5464af9bcbd68d8fb9877d4; no voters:
I20260806 22:47:22.445869 16763 raft_consensus.cc:2781] T f28ae5f6eeac4a6b8200bf1976754289 P 62e43f48d5464af9bcbd68d8fb9877d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:47:22.446053 16763 raft_consensus.cc:686] T f28ae5f6eeac4a6b8200bf1976754289 P 62e43f48d5464af9bcbd68d8fb9877d4 [term 1 LEADER]: Becoming Leader. State: Replica: 62e43f48d5464af9bcbd68d8fb9877d4, State: Running, Role: LEADER
I20260806 22:47:22.446440 16763 consensus_queue.cc:237] T f28ae5f6eeac4a6b8200bf1976754289 P 62e43f48d5464af9bcbd68d8fb9877d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1fec59295474125ae87691d06eaf345" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 34829 } } peers { permanent_uuid: "62e43f48d5464af9bcbd68d8fb9877d4" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45455 } } peers { permanent_uuid: "35a7f06767bb4b81999ee9fa87d12aa3" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 43189 } }
I20260806 22:47:22.446502 16731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:22.446387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:38077 (local address 127.6.40.67:54417)
0806 22:47:22.446446 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:22.446479 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:38077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260806 22:47:22.447341 15986 catalog_manager.cc:5495] T f28ae5f6eeac4a6b8200bf1976754289 P 62e43f48d5464af9bcbd68d8fb9877d4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 62e43f48d5464af9bcbd68d8fb9877d4 (127.6.40.67). New cstate: current_term: 1 leader_uuid: "62e43f48d5464af9bcbd68d8fb9877d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1fec59295474125ae87691d06eaf345" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 34829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62e43f48d5464af9bcbd68d8fb9877d4" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45455 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "35a7f06767bb4b81999ee9fa87d12aa3" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 43189 } health_report { overall_health: UNKNOWN } } }
I20260806 22:47:22.473292 16751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:22.470750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58330 (local address 127.6.40.67:45455)
0806 22:47:22.470823 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:22.470825 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:22.470834 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:47:22.470879 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:22.470880 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:22.470904 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:22.470967 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:47:22.471080 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:22.471566 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:22.472336 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:22.472545 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:22.472606 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:22.472801 (+ 195us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:47:22.472977 (+ 176us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:47:22.473037 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:22.473151 (+ 114us) server_negotiation.cc:299] Negotiation successful
0806 22:47:22.473173 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 22:47:22.474686 16183 raft_consensus.cc:1261] T f28ae5f6eeac4a6b8200bf1976754289 P 35a7f06767bb4b81999ee9fa87d12aa3 [term 1 FOLLOWER]: Refusing update from remote peer 62e43f48d5464af9bcbd68d8fb9877d4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:47:22.474829 16366 raft_consensus.cc:1261] T f28ae5f6eeac4a6b8200bf1976754289 P d1fec59295474125ae87691d06eaf345 [term 1 FOLLOWER]: Refusing update from remote peer 62e43f48d5464af9bcbd68d8fb9877d4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:47:22.475064 16763 consensus_queue.cc:1035] T f28ae5f6eeac4a6b8200bf1976754289 P 62e43f48d5464af9bcbd68d8fb9877d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35a7f06767bb4b81999ee9fa87d12aa3" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 43189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:47:22.475225 16767 consensus_queue.cc:1035] T f28ae5f6eeac4a6b8200bf1976754289 P 62e43f48d5464af9bcbd68d8fb9877d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1fec59295474125ae87691d06eaf345" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 34829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:47:22.476078 16732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:22.475951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:38077 (local address 127.6.40.65:41961)
0806 22:47:22.476000 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:22.476036 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:38077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260806 22:47:22.476611 16730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:22.476464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:38077 (local address 127.6.40.66:36797)
0806 22:47:22.476544 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:22.476574 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:38077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260806 22:47:22.477591 16770 mvcc.cc:205] Tried to move back new op lower bound from 7315687188373667840 to 7315687188261539840. Current Snapshot: MvccSnapshot[applied={T|T < 7315687188373667840}]
I20260806 22:47:22.477802 16731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:22.477686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:38077 (local address 127.6.40.67:47051)
0806 22:47:22.477740 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:22.477775 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:38077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260806 22:47:22.478022 16771 mvcc.cc:205] Tried to move back new op lower bound from 7315687188373667840 to 7315687188261539840. Current Snapshot: MvccSnapshot[applied={T|T < 7315687188373667840}]
I20260806 22:47:22.478368 16732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:22.478219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:38077 (local address 127.6.40.65:43989)
0806 22:47:22.478300 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:22.478325 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:38077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260806 22:47:22.478456 16775 mvcc.cc:205] Tried to move back new op lower bound from 7315687188373667840 to 7315687188261539840. Current Snapshot: MvccSnapshot[applied={T|T < 7315687188373667840}]
I20260806 22:47:22.478561 16730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:22.478201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:38077 (local address 127.6.40.66:52575)
0806 22:47:22.478500 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:22.478534 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:38077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":278}
I20260806 22:47:22.480751 16731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:22.480651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:38077 (local address 127.6.40.67:34259)
0806 22:47:22.480689 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:22.480719 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:38077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260806 22:47:22.480947 15986 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56788:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:47:22.481063 15986 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56788:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:47:22.482595 15986 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cca9c82ca527425abfcddb8d93dd4b64: Sending DeleteTablet for 3 replicas of tablet f28ae5f6eeac4a6b8200bf1976754289
I20260806 22:47:22.483016 16532 tablet_service.cc:1514] Processing DeleteTablet for tablet f28ae5f6eeac4a6b8200bf1976754289 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:47:22 UTC) from {username='oryx', principal='oryx/127.6.40.124@KRBTEST.COM'} at 127.0.0.1:58324
I20260806 22:47:22.483065 16338 tablet_service.cc:1514] Processing DeleteTablet for tablet f28ae5f6eeac4a6b8200bf1976754289 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:47:22 UTC) from {username='oryx', principal='oryx/127.6.40.124@KRBTEST.COM'} at 127.0.0.1:47980
I20260806 22:47:22.483165 16158 tablet_service.cc:1514] Processing DeleteTablet for tablet f28ae5f6eeac4a6b8200bf1976754289 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:47:22 UTC) from {username='oryx', principal='oryx/127.6.40.124@KRBTEST.COM'} at 127.0.0.1:47214
I20260806 22:47:22.483291 16786 tablet_replica.cc:330] T f28ae5f6eeac4a6b8200bf1976754289 P d1fec59295474125ae87691d06eaf345: stopping tablet replica
I20260806 22:47:22.483363 16786 raft_consensus.cc:2229] T f28ae5f6eeac4a6b8200bf1976754289 P d1fec59295474125ae87691d06eaf345 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:47:22.483429 16786 raft_consensus.cc:2258] T f28ae5f6eeac4a6b8200bf1976754289 P d1fec59295474125ae87691d06eaf345 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:47:22.483632 16787 tablet_replica.cc:330] T f28ae5f6eeac4a6b8200bf1976754289 P 35a7f06767bb4b81999ee9fa87d12aa3: stopping tablet replica
I20260806 22:47:22.483703 16787 raft_consensus.cc:2229] T f28ae5f6eeac4a6b8200bf1976754289 P 35a7f06767bb4b81999ee9fa87d12aa3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:47:22.483754 16787 raft_consensus.cc:2258] T f28ae5f6eeac4a6b8200bf1976754289 P 35a7f06767bb4b81999ee9fa87d12aa3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:47:22.483855 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 15945
I20260806 22:47:22.489691 16786 ts_tablet_manager.cc:1905] T f28ae5f6eeac4a6b8200bf1976754289 P d1fec59295474125ae87691d06eaf345: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:47:22.491315 16786 ts_tablet_manager.cc:1918] T f28ae5f6eeac4a6b8200bf1976754289 P d1fec59295474125ae87691d06eaf345: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:47:22.491393 16786 log.cc:1196] T f28ae5f6eeac4a6b8200bf1976754289 P d1fec59295474125ae87691d06eaf345: Deleting WAL directory at /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/ts-1/wal/wals/f28ae5f6eeac4a6b8200bf1976754289
I20260806 22:47:22.491650 16786 ts_tablet_manager.cc:1939] T f28ae5f6eeac4a6b8200bf1976754289 P d1fec59295474125ae87691d06eaf345: Deleting consensus metadata
I20260806 22:47:22.496665 15979 catalog_manager.cc:4841] TS d1fec59295474125ae87691d06eaf345 (127.6.40.66:34829): tablet f28ae5f6eeac4a6b8200bf1976754289 (table test-table [id=b9f560526246486dbd27142a233f4f77]) successfully deleted
I20260806 22:47:22.496757 16785 tablet_replica.cc:330] T f28ae5f6eeac4a6b8200bf1976754289 P 62e43f48d5464af9bcbd68d8fb9877d4: stopping tablet replica
I20260806 22:47:22.496879 16785 raft_consensus.cc:2229] T f28ae5f6eeac4a6b8200bf1976754289 P 62e43f48d5464af9bcbd68d8fb9877d4 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:47:22.497028 16785 raft_consensus.cc:2258] T f28ae5f6eeac4a6b8200bf1976754289 P 62e43f48d5464af9bcbd68d8fb9877d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:47:22.497428 16785 ts_tablet_manager.cc:1905] T f28ae5f6eeac4a6b8200bf1976754289 P 62e43f48d5464af9bcbd68d8fb9877d4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:47:22.499620 16785 ts_tablet_manager.cc:1918] T f28ae5f6eeac4a6b8200bf1976754289 P 62e43f48d5464af9bcbd68d8fb9877d4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:47:22.499702 16785 log.cc:1196] T f28ae5f6eeac4a6b8200bf1976754289 P 62e43f48d5464af9bcbd68d8fb9877d4: Deleting WAL directory at /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056405724609-6305-0/minicluster-data/ts-2/wal/wals/f28ae5f6eeac4a6b8200bf1976754289
I20260806 22:47:22.499977 16785 ts_tablet_manager.cc:1939] T f28ae5f6eeac4a6b8200bf1976754289 P 62e43f48d5464af9bcbd68d8fb9877d4: Deleting consensus metadata
I20260806 22:47:22.500499 15978 catalog_manager.cc:4841] TS 62e43f48d5464af9bcbd68d8fb9877d4 (127.6.40.67:45455): tablet f28ae5f6eeac4a6b8200bf1976754289 (table test-table [id=b9f560526246486dbd27142a233f4f77]) successfully deleted
I20260806 22:47:22.503319 16748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:22.503167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.65:43189 (local address 127.0.0.1:47224)
0806 22:47:22.503239 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:22.503277 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.65:43189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260806 22:47:22.503521 15978 catalog_manager.cc:4584] TS 35a7f06767bb4b81999ee9fa87d12aa3 (127.6.40.65:43189): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f28ae5f6eeac4a6b8200bf1976754289: Network error: Client connection negotiation failed: client connection to 127.6.40.65:43189: connect: Connection refused (error 111)
I20260806 22:47:22.503614 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 16259
I20260806 22:47:22.519119 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 16443
I20260806 22:47:22.528946 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 15567
I20260806 22:47:22.536831 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 15819
2026-08-06T22:47:22Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (6851 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:47:22 dist-test-slave-zh2d krb5kdc[16793](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:47:22 dist-test-slave-zh2d krb5kdc[16793](info): set up 2 sockets
Aug 06 22:47:22 dist-test-slave-zh2d krb5kdc[16793](info): commencing operation
krb5kdc: starting...
W20260806 22:47:23.765538 6305 mini_kdc.cc:121] Time spent starting KDC: real 1.200s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/krb5kdc/test-user.keytab.
Aug 06 22:47:23 dist-test-slave-zh2d krb5kdc[16793](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056443, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:47:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:47:23Z Disabled control of system clock
WARNING: no policy specified for oryx/127.6.40.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.40.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.40.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/krb5kdc/oryx_127.6.40.126.keytab.
Entry for principal oryx/127.6.40.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/krb5kdc/oryx_127.6.40.126.keytab.
I20260806 22:47:23.892668 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.126:39333
--webserver_interface=127.6.40.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/krb5kdc/oryx_127.6.40.126.keytab
--principal=oryx/127.6.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:34281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.40.126:39333
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:47:23.978420 16810 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:47:23.978633 16810 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:47:23.978655 16810 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:47:23.979966 16810 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:47:23.980018 16810 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 22:47:23.980032 16810 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:47:23.980044 16810 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:47:23.980057 16810 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:47:23.980069 16810 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:47:23.981700 16810 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:34281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.40.126:39333
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/krb5kdc/oryx_127.6.40.126.keytab
--principal=oryx/127.6.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.126:39333
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16810
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:47:23.981926 16810 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:47:23.982115 16810 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:47:23.987524 16822 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:23.987537 16819 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:23.987530 16817 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:23.987547 16810 server_base.cc:1034] running on GCE node
I20260806 22:47:23.988180 16810 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:47:23.988494 16810 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:47:23.989675 16810 hybrid_clock.cc:648] HybridClock initialized: now 1786056443989655 us; error 51 us; skew 500 ppm
Aug 06 22:47:23 dist-test-slave-zh2d krb5kdc[16793](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056443, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:47:23.992389 16810 init.cc:377] Logged in from keytab as oryx/127.6.40.126@KRBTEST.COM (short username oryx)
I20260806 22:47:23.993010 16810 webserver.cc:460] Webserver started at http://127.6.40.126:45141/ using document root <none> and password file <none>
I20260806 22:47:23.993299 16810 fs_manager.cc:359] Metadata directory not provided
I20260806 22:47:23.993382 16810 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:47:23.993551 16810 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:47:23.994755 16810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/minicluster-data/master-0/data/instance:
uuid: "a023cd9df99d4e3db64a1b4977884980"
format_stamp: "Formatted at 2026-08-06 22:47:23 on dist-test-slave-zh2d"
server_key: "3feddcb76ae1b645611869b6b0e2bd5f"
server_key_iv: "45959a5feedb4b7dd52da359d8e9642b"
server_key_version: "encryptionkey@0"
I20260806 22:47:23.995180 16810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/minicluster-data/master-0/wal/instance:
uuid: "a023cd9df99d4e3db64a1b4977884980"
format_stamp: "Formatted at 2026-08-06 22:47:23 on dist-test-slave-zh2d"
server_key: "3feddcb76ae1b645611869b6b0e2bd5f"
server_key_iv: "45959a5feedb4b7dd52da359d8e9642b"
server_key_version: "encryptionkey@0"
I20260806 22:47:23.996744 16810 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:47:23.997718 16833 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:23.997987 16810 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:47:23.998116 16810 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/minicluster-data/master-0/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/minicluster-data/master-0/wal
uuid: "a023cd9df99d4e3db64a1b4977884980"
format_stamp: "Formatted at 2026-08-06 22:47:23 on dist-test-slave-zh2d"
server_key: "3feddcb76ae1b645611869b6b0e2bd5f"
server_key_iv: "45959a5feedb4b7dd52da359d8e9642b"
server_key_version: "encryptionkey@0"
I20260806 22:47:23.998206 16810 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:47:24.026907 16810 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:47:24.029737 16810 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:47:24.029909 16810 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:47:24.035475 16810 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.126:39333
I20260806 22:47:24.035578 16925 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.126:39333 every 8 connection(s)
I20260806 22:47:24.035966 16810 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
I20260806 22:47:24.036779 16927 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:47:24.039256 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 16810
I20260806 22:47:24.039386 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/minicluster-data/master-0/wal/instance
I20260806 22:47:24.039377 16927 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a023cd9df99d4e3db64a1b4977884980: Bootstrap starting.
I20260806 22:47:24.039608 6305 external_mini_cluster.cc:1428] Setting key 15c7f69d40cb9c6f4b32439c9ac89775
I20260806 22:47:24.040197 16927 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a023cd9df99d4e3db64a1b4977884980: Neither blocks nor log segments found. Creating new log.
I20260806 22:47:24.040586 16927 log.cc:824] T 00000000000000000000000000000000 P a023cd9df99d4e3db64a1b4977884980: Log is configured to *not* fsync() on all Append() calls
I20260806 22:47:24.041465 16927 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a023cd9df99d4e3db64a1b4977884980: No bootstrap required, opened a new log
I20260806 22:47:24.043977 16927 raft_consensus.cc:348] T 00000000000000000000000000000000 P a023cd9df99d4e3db64a1b4977884980 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a023cd9df99d4e3db64a1b4977884980" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 39333 } }
I20260806 22:47:24.044149 16927 raft_consensus.cc:374] T 00000000000000000000000000000000 P a023cd9df99d4e3db64a1b4977884980 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:47:24.044198 16927 raft_consensus.cc:729] T 00000000000000000000000000000000 P a023cd9df99d4e3db64a1b4977884980 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a023cd9df99d4e3db64a1b4977884980, State: Initialized, Role: FOLLOWER
I20260806 22:47:24.044370 16927 consensus_queue.cc:260] T 00000000000000000000000000000000 P a023cd9df99d4e3db64a1b4977884980 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a023cd9df99d4e3db64a1b4977884980" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 39333 } }
I20260806 22:47:24.044466 16927 raft_consensus.cc:388] T 00000000000000000000000000000000 P a023cd9df99d4e3db64a1b4977884980 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:47:24.044500 16927 raft_consensus.cc:482] T 00000000000000000000000000000000 P a023cd9df99d4e3db64a1b4977884980 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:47:24.044539 16927 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a023cd9df99d4e3db64a1b4977884980 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:47:24.045296 16927 raft_consensus.cc:504] T 00000000000000000000000000000000 P a023cd9df99d4e3db64a1b4977884980 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a023cd9df99d4e3db64a1b4977884980" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 39333 } }
I20260806 22:47:24.045401 16927 leader_election.cc:302] T 00000000000000000000000000000000 P a023cd9df99d4e3db64a1b4977884980 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a023cd9df99d4e3db64a1b4977884980; no voters:
I20260806 22:47:24.045595 16927 leader_election.cc:288] T 00000000000000000000000000000000 P a023cd9df99d4e3db64a1b4977884980 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:47:24.045620 16934 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a023cd9df99d4e3db64a1b4977884980 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:47:24.045743 16934 raft_consensus.cc:686] T 00000000000000000000000000000000 P a023cd9df99d4e3db64a1b4977884980 [term 1 LEADER]: Becoming Leader. State: Replica: a023cd9df99d4e3db64a1b4977884980, State: Running, Role: LEADER
I20260806 22:47:24.045837 16934 consensus_queue.cc:237] T 00000000000000000000000000000000 P a023cd9df99d4e3db64a1b4977884980 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a023cd9df99d4e3db64a1b4977884980" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 39333 } }
I20260806 22:47:24.045876 16927 sys_catalog.cc:564] T 00000000000000000000000000000000 P a023cd9df99d4e3db64a1b4977884980 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:47:24.046180 16934 sys_catalog.cc:455] T 00000000000000000000000000000000 P a023cd9df99d4e3db64a1b4977884980 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a023cd9df99d4e3db64a1b4977884980" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a023cd9df99d4e3db64a1b4977884980" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 39333 } } }
I20260806 22:47:24.046253 16934 sys_catalog.cc:458] T 00000000000000000000000000000000 P a023cd9df99d4e3db64a1b4977884980 [sys.catalog]: This master's current role is: LEADER
I20260806 22:47:24.046394 16934 sys_catalog.cc:455] T 00000000000000000000000000000000 P a023cd9df99d4e3db64a1b4977884980 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a023cd9df99d4e3db64a1b4977884980. Latest consensus state: current_term: 1 leader_uuid: "a023cd9df99d4e3db64a1b4977884980" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a023cd9df99d4e3db64a1b4977884980" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 39333 } } }
I20260806 22:47:24.046466 16934 sys_catalog.cc:458] T 00000000000000000000000000000000 P a023cd9df99d4e3db64a1b4977884980 [sys.catalog]: This master's current role is: LEADER
I20260806 22:47:24.046614 16942 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:47:24.047416 16942 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 06 22:47:24 dist-test-slave-zh2d krb5kdc[16793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056443, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM
I20260806 22:47:24.049400 16942 catalog_manager.cc:1347] Generated new cluster ID: 4cab02fd7411432f93ebc5daf98f9fff
I20260806 22:47:24.049455 16942 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:47:24.055312 16942 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:47:24.055788 16931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:24.040212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59676 (local address 127.6.40.126:39333)
0806 22:47:24.040508 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:24.040518 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:24.040553 (+ 35us) server_negotiation.cc:407] Connection header received
0806 22:47:24.041126 (+ 573us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:24.041136 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:24.041205 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:24.041370 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:24.042072 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:24.043580 (+ 1508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:24.044435 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:24.045084 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:24.045216 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:24.048181 (+ 2965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:24.048186 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:24.048187 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:24.048200 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:24.053886 (+ 5686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:24.054378 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:24.054381 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:24.054381 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:24.054471 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:24.054704 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:24.054705 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:24.054705 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:24.054935 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:24.055108 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:24.055480 (+ 372us) server_negotiation.cc:299] Negotiation successful
0806 22:47:24.055623 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":241,"thread_start_us":85,"threads_started":1}
I20260806 22:47:24.056252 16942 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:47:24.072419 16942 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a023cd9df99d4e3db64a1b4977884980: Generated new TSK 0
I20260806 22:47:24.072669 16942 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.6.40.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.40.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.40.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/krb5kdc/oryx_127.6.40.65.keytab.
Entry for principal oryx/127.6.40.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/krb5kdc/oryx_127.6.40.65.keytab.
I20260806 22:47:24.112036 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.65:0
--local_ip_for_outbound_sockets=127.6.40.65
--webserver_interface=127.6.40.65
--webserver_port=0
--tserver_master_addrs=127.6.40.126:39333
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/krb5kdc/oryx_127.6.40.65.keytab
--principal=oryx/127.6.40.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:34281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:47:24.206163 16961 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:47:24.206375 16961 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:47:24.206409 16961 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:47:24.207650 16961 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:47:24.207696 16961 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:47:24.207718 16961 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 22:47:24.207737 16961 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.65
I20260806 22:47:24.209308 16961 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:34281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/krb5kdc/oryx_127.6.40.65.keytab
--principal=oryx/127.6.40.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.40.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.40.126:39333
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16961
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.65
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:47:24.209509 16961 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:47:24.209699 16961 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:47:24.214939 16968 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:24.215044 16961 server_base.cc:1034] running on GCE node
W20260806 22:47:24.215005 16971 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:24.214890 16967 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:24.215312 16961 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:47:24.215536 16961 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:47:24.216719 16961 hybrid_clock.cc:648] HybridClock initialized: now 1786056444216712 us; error 40 us; skew 500 ppm
Aug 06 22:47:24 dist-test-slave-zh2d krb5kdc[16793](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056444, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.40.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:47:24.219429 16961 init.cc:377] Logged in from keytab as oryx/127.6.40.65@KRBTEST.COM (short username oryx)
I20260806 22:47:24.219918 16961 webserver.cc:460] Webserver started at http://127.6.40.65:46071/ using document root <none> and password file <none>
I20260806 22:47:24.220146 16961 fs_manager.cc:359] Metadata directory not provided
I20260806 22:47:24.220214 16961 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:47:24.220368 16961 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:47:24.221524 16961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/minicluster-data/ts-0/data/instance:
uuid: "1a3c36e213304396bf7ebddea9300695"
format_stamp: "Formatted at 2026-08-06 22:47:24 on dist-test-slave-zh2d"
server_key: "bef1edb96a3c43f77e2962c78af1bcc3"
server_key_iv: "ed4e9793496bd47be699a9f8b4996ffe"
server_key_version: "encryptionkey@0"
I20260806 22:47:24.221899 16961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/minicluster-data/ts-0/wal/instance:
uuid: "1a3c36e213304396bf7ebddea9300695"
format_stamp: "Formatted at 2026-08-06 22:47:24 on dist-test-slave-zh2d"
server_key: "bef1edb96a3c43f77e2962c78af1bcc3"
server_key_iv: "ed4e9793496bd47be699a9f8b4996ffe"
server_key_version: "encryptionkey@0"
I20260806 22:47:24.223289 16961 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 22:47:24.224061 16980 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:24.224232 16961 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:24.224342 16961 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/minicluster-data/ts-0/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/minicluster-data/ts-0/wal
uuid: "1a3c36e213304396bf7ebddea9300695"
format_stamp: "Formatted at 2026-08-06 22:47:24 on dist-test-slave-zh2d"
server_key: "bef1edb96a3c43f77e2962c78af1bcc3"
server_key_iv: "ed4e9793496bd47be699a9f8b4996ffe"
server_key_version: "encryptionkey@0"
I20260806 22:47:24.224452 16961 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:47:24.240545 16961 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:47:24.242800 16961 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:47:24.242976 16961 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:47:24.243606 16961 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:47:24.243666 16961 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:24.243708 16961 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:47:24.243731 16961 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 22:47:24 dist-test-slave-zh2d krb5kdc[16793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056444, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.40.65@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM
I20260806 22:47:24.251746 16961 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.65:38371
I20260806 22:47:24.252146 16961 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
I20260806 22:47:24.252581 17153 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.65:38371 every 8 connection(s)
I20260806 22:47:24.254910 16931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:24.244048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.65:55073 (local address 127.6.40.126:39333)
0806 22:47:24.244121 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:24.244123 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:24.244400 (+ 277us) server_negotiation.cc:407] Connection header received
0806 22:47:24.245199 (+ 799us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:24.245199 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:24.245210 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:24.245258 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:24.246289 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:24.247107 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:24.248974 (+ 1867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:24.249213 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:24.249280 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:24.252166 (+ 2886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:24.252168 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:24.252168 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:24.252178 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:24.253736 (+ 1558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:24.254123 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:24.254124 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:24.254124 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:24.254177 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:24.254355 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:24.254356 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:24.254356 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:24.254460 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:24.254511 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:24.254750 (+ 239us) server_negotiation.cc:299] Negotiation successful
0806 22:47:24.254827 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 22:47:24.255077 16999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:24.244030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:39333 (local address 127.6.40.65:55073)
0806 22:47:24.244231 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:24.244285 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:24.245087 (+ 802us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:24.245356 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:24.245362 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:24.245831 (+ 469us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:24.246172 (+ 341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:24.246179 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:24.247187 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:24.247188 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:24.248828 (+ 1640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:24.248832 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:24.249302 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:24.249303 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:24.249473 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:24.250496 (+ 1023us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:24.250504 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:24.252059 (+ 1555us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:47:24.253813 (+ 1754us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:24.253820 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:24.253822 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:24.254053 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:24.254219 (+ 166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:24.254220 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:24.254220 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:24.254306 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:24.254519 (+ 213us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:47:24.254525 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:47:24.254640 (+ 115us) client_negotiation.cc:769] Sending connection context
0806 22:47:24.254723 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 22:47:24.254846 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":102,"threads_started":1}
I20260806 22:47:24.255636 17154 heartbeater.cc:347] Connected to a master server at 127.6.40.126:39333
I20260806 22:47:24.255723 17154 heartbeater.cc:464] Registering TS with master...
I20260806 22:47:24.255935 17154 heartbeater.cc:511] Master 127.6.40.126:39333 requested a full tablet report, sending...
I20260806 22:47:24.256567 16858 ts_manager.cc:194] Registered new tserver with Master: 1a3c36e213304396bf7ebddea9300695 (127.6.40.65:38371)
I20260806 22:47:24.257469 16858 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.40.65@KRBTEST.COM'} at 127.6.40.65:55073
I20260806 22:47:24.258244 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 16961
I20260806 22:47:24.258361 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/minicluster-data/ts-0/wal/instance
I20260806 22:47:24.258581 6305 external_mini_cluster.cc:1428] Setting key 94dbc793401669dd540348eda0db96e9
I20260806 22:47:24.262763 16931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:24.256716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.65:40313 (local address 127.6.40.126:39333)
0806 22:47:24.256755 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:24.256756 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:24.257023 (+ 267us) server_negotiation.cc:407] Connection header received
0806 22:47:24.257048 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:24.257049 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:24.257060 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:24.257120 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:24.257416 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:24.258088 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:24.258834 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:24.259116 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:24.259184 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:24.260151 (+ 967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:24.260152 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:24.260153 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:24.260160 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:24.261632 (+ 1472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:24.262043 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:24.262044 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:24.262044 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:24.262094 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:24.262301 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:24.262302 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:24.262302 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:24.262395 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:24.262483 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:24.262647 (+ 164us) server_negotiation.cc:299] Negotiation successful
0806 22:47:24.262703 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 22:47:24.262900 17166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:24.256646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:39333 (local address 127.6.40.65:40313)
0806 22:47:24.256950 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:24.256959 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:24.257014 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:24.257131 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:24.257131 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:24.257294 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:24.257348 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:24.257349 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:24.258200 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:24.258201 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:24.258770 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:24.258772 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:24.259199 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:24.259200 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:24.259269 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:24.259764 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:24.259770 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:24.260059 (+ 289us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:47:24.261720 (+ 1661us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:24.261721 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:24.261722 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:24.261968 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:24.262165 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:24.262165 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:24.262166 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:24.262240 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:24.262497 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:47:24.262498 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:47:24.262549 (+ 51us) client_negotiation.cc:769] Sending connection context
0806 22:47:24.262627 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 22:47:24.262748 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":70,"threads_started":1}
WARNING: no policy specified for oryx/127.6.40.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.40.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.40.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/krb5kdc/oryx_127.6.40.66.keytab.
Entry for principal oryx/127.6.40.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/krb5kdc/oryx_127.6.40.66.keytab.
I20260806 22:47:24.296981 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.66:0
--local_ip_for_outbound_sockets=127.6.40.66
--webserver_interface=127.6.40.66
--webserver_port=0
--tserver_master_addrs=127.6.40.126:39333
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/krb5kdc/oryx_127.6.40.66.keytab
--principal=oryx/127.6.40.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:34281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:47:24.382103 17169 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:47:24.382302 17169 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:47:24.382325 17169 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:47:24.383399 17169 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:47:24.383433 17169 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:47:24.383453 17169 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 22:47:24.383471 17169 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.66
I20260806 22:47:24.385136 17169 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:34281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/krb5kdc/oryx_127.6.40.66.keytab
--principal=oryx/127.6.40.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.40.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.40.126:39333
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17169
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.66
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:47:24.385332 17169 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:47:24.385509 17169 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:47:24.390699 17175 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:24.390750 17176 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:24.390779 17169 server_base.cc:1034] running on GCE node
W20260806 22:47:24.390750 17178 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:24.391134 17169 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:47:24.391340 17169 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:47:24.392518 17169 hybrid_clock.cc:648] HybridClock initialized: now 1786056444392505 us; error 34 us; skew 500 ppm
Aug 06 22:47:24 dist-test-slave-zh2d krb5kdc[16793](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056444, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.40.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:47:24.395016 17169 init.cc:377] Logged in from keytab as oryx/127.6.40.66@KRBTEST.COM (short username oryx)
I20260806 22:47:24.395411 17169 webserver.cc:460] Webserver started at http://127.6.40.66:45313/ using document root <none> and password file <none>
I20260806 22:47:24.395588 17169 fs_manager.cc:359] Metadata directory not provided
I20260806 22:47:24.395628 17169 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:47:24.395716 17169 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:47:24.396725 17169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/minicluster-data/ts-1/data/instance:
uuid: "a712a3a80999468ea0a596422fb8f3a1"
format_stamp: "Formatted at 2026-08-06 22:47:24 on dist-test-slave-zh2d"
server_key: "54f625e822e6f1701067039ea1dd07e7"
server_key_iv: "ca048fa73aba395d375ece5972098890"
server_key_version: "encryptionkey@0"
I20260806 22:47:24.397115 17169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/minicluster-data/ts-1/wal/instance:
uuid: "a712a3a80999468ea0a596422fb8f3a1"
format_stamp: "Formatted at 2026-08-06 22:47:24 on dist-test-slave-zh2d"
server_key: "54f625e822e6f1701067039ea1dd07e7"
server_key_iv: "ca048fa73aba395d375ece5972098890"
server_key_version: "encryptionkey@0"
I20260806 22:47:24.398475 17169 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:47:24.399227 17187 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:24.399431 17169 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:47:24.399519 17169 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/minicluster-data/ts-1/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/minicluster-data/ts-1/wal
uuid: "a712a3a80999468ea0a596422fb8f3a1"
format_stamp: "Formatted at 2026-08-06 22:47:24 on dist-test-slave-zh2d"
server_key: "54f625e822e6f1701067039ea1dd07e7"
server_key_iv: "ca048fa73aba395d375ece5972098890"
server_key_version: "encryptionkey@0"
I20260806 22:47:24.399573 17169 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:47:24.414983 17169 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:47:24.417344 17169 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:47:24.417460 17169 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:47:24.417943 17169 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:47:24.417981 17169 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:24.418004 17169 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:47:24.418017 17169 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:24.425151 17169 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.66:44463
I20260806 22:47:24.425269 17352 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.66:44463 every 8 connection(s)
Aug 06 22:47:24 dist-test-slave-zh2d krb5kdc[16793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056444, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.40.66@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM
I20260806 22:47:24.425545 17169 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
I20260806 22:47:24.428747 16931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:24.418599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.66:56177 (local address 127.6.40.126:39333)
0806 22:47:24.418679 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:24.418682 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:24.418838 (+ 156us) server_negotiation.cc:407] Connection header received
0806 22:47:24.419371 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:24.419373 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:24.419389 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:24.419472 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:24.420398 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:24.421279 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:24.422865 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:24.423191 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:24.423281 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:24.426080 (+ 2799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:24.426081 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:24.426081 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:24.426089 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:24.427494 (+ 1405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:24.427865 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:24.427866 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:24.427866 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:24.427922 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:24.428144 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:24.428144 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:24.428144 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:24.428247 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:24.428300 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:24.428608 (+ 308us) server_negotiation.cc:299] Negotiation successful
0806 22:47:24.428678 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 22:47:24.428983 17198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:24.418514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:39333 (local address 127.6.40.66:56177)
0806 22:47:24.418701 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:24.418743 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:24.419248 (+ 505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:24.419549 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:24.419552 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:24.419948 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:24.420276 (+ 328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:24.420284 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:24.421391 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:24.421392 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:24.422765 (+ 1373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:24.422767 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:24.423286 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:24.423287 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:24.423388 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:24.423938 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:24.423950 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:24.425988 (+ 2038us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:47:24.427566 (+ 1578us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:24.427573 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:24.427574 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:24.427794 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:24.427985 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:24.427986 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:24.427986 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:24.428089 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:24.428328 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:47:24.428335 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:47:24.428482 (+ 147us) client_negotiation.cc:769] Sending connection context
0806 22:47:24.428578 (+ 96us) client_negotiation.cc:240] Negotiation successful
0806 22:47:24.428706 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":97,"threads_started":1}
I20260806 22:47:24.429584 17354 heartbeater.cc:347] Connected to a master server at 127.6.40.126:39333
I20260806 22:47:24.429677 17354 heartbeater.cc:464] Registering TS with master...
I20260806 22:47:24.429891 17354 heartbeater.cc:511] Master 127.6.40.126:39333 requested a full tablet report, sending...
I20260806 22:47:24.430373 16858 ts_manager.cc:194] Registered new tserver with Master: a712a3a80999468ea0a596422fb8f3a1 (127.6.40.66:44463)
I20260806 22:47:24.430882 16858 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.40.66@KRBTEST.COM'} at 127.6.40.66:56177
I20260806 22:47:24.433015 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 17169
I20260806 22:47:24.433149 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/minicluster-data/ts-1/wal/instance
I20260806 22:47:24.433408 6305 external_mini_cluster.cc:1428] Setting key 7edc0fc208ccdb5a3a4d29b48bf72dcd
I20260806 22:47:24.437000 16931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:24.430831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.66:54787 (local address 127.6.40.126:39333)
0806 22:47:24.430880 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:24.430882 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:24.430993 (+ 111us) server_negotiation.cc:407] Connection header received
0806 22:47:24.431094 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:24.431095 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:24.431113 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:24.431192 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:24.431591 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:24.432109 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:24.432811 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:24.433104 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:24.433187 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:24.434198 (+ 1011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:24.434200 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:24.434200 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:24.434207 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:24.435615 (+ 1408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:24.436049 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:24.436050 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:24.436050 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:24.436117 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:24.436325 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:24.436326 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:24.436326 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:24.436520 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:24.436607 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:24.436776 (+ 169us) server_negotiation.cc:299] Negotiation successful
0806 22:47:24.436906 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 22:47:24.437016 17360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:24.430728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:39333 (local address 127.6.40.66:54787)
0806 22:47:24.430920 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:24.430932 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:24.431021 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:24.431191 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:24.431192 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:24.431435 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:24.431501 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:24.431503 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:24.432205 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:24.432206 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:24.432702 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:24.432703 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:24.433210 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:24.433212 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:24.433284 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:24.433790 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:24.433794 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:24.434084 (+ 290us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:47:24.435734 (+ 1650us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:24.435735 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:24.435736 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:24.435956 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:24.436187 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:24.436188 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:24.436189 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:24.436256 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:24.436597 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:47:24.436598 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:47:24.436652 (+ 54us) client_negotiation.cc:769] Sending connection context
0806 22:47:24.436755 (+ 103us) client_negotiation.cc:240] Negotiation successful
0806 22:47:24.436903 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":168,"threads_started":1}
WARNING: no policy specified for oryx/127.6.40.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.40.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.40.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/krb5kdc/oryx_127.6.40.67.keytab.
Entry for principal oryx/127.6.40.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/krb5kdc/oryx_127.6.40.67.keytab.
I20260806 22:47:24.464598 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.67:0
--local_ip_for_outbound_sockets=127.6.40.67
--webserver_interface=127.6.40.67
--webserver_port=0
--tserver_master_addrs=127.6.40.126:39333
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/krb5kdc/oryx_127.6.40.67.keytab
--principal=oryx/127.6.40.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:34281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:47:24.551151 17363 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:47:24.551357 17363 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:47:24.551383 17363 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:47:24.552570 17363 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:47:24.552614 17363 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:47:24.552634 17363 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 22:47:24.552652 17363 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.67
I20260806 22:47:24.554304 17363 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:34281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/krb5kdc/oryx_127.6.40.67.keytab
--principal=oryx/127.6.40.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.40.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.40.126:39333
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17363
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.67
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:47:24.554570 17363 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:47:24.554812 17363 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:47:24.560252 17369 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:24.560252 17370 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:24.560559 17363 server_base.cc:1034] running on GCE node
W20260806 22:47:24.560273 17372 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:24.560905 17363 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:47:24.561136 17363 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:47:24.562347 17363 hybrid_clock.cc:648] HybridClock initialized: now 1786056444562320 us; error 52 us; skew 500 ppm
Aug 06 22:47:24 dist-test-slave-zh2d krb5kdc[16793](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056444, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.40.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:47:24.565223 17363 init.cc:377] Logged in from keytab as oryx/127.6.40.67@KRBTEST.COM (short username oryx)
I20260806 22:47:24.565761 17363 webserver.cc:460] Webserver started at http://127.6.40.67:34937/ using document root <none> and password file <none>
I20260806 22:47:24.566027 17363 fs_manager.cc:359] Metadata directory not provided
I20260806 22:47:24.566102 17363 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:47:24.566257 17363 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:47:24.567371 17363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/minicluster-data/ts-2/data/instance:
uuid: "4091dc731c9a4e15b38508539576d38e"
format_stamp: "Formatted at 2026-08-06 22:47:24 on dist-test-slave-zh2d"
server_key: "49792fedbedaf27bd5626d007d693a04"
server_key_iv: "d9d6c7db74e270cf024bd590a4fbcebf"
server_key_version: "encryptionkey@0"
I20260806 22:47:24.567744 17363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/minicluster-data/ts-2/wal/instance:
uuid: "4091dc731c9a4e15b38508539576d38e"
format_stamp: "Formatted at 2026-08-06 22:47:24 on dist-test-slave-zh2d"
server_key: "49792fedbedaf27bd5626d007d693a04"
server_key_iv: "d9d6c7db74e270cf024bd590a4fbcebf"
server_key_version: "encryptionkey@0"
I20260806 22:47:24.569216 17363 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.002s
I20260806 22:47:24.570014 17381 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:24.570214 17363 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:47:24.570328 17363 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/minicluster-data/ts-2/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/minicluster-data/ts-2/wal
uuid: "4091dc731c9a4e15b38508539576d38e"
format_stamp: "Formatted at 2026-08-06 22:47:24 on dist-test-slave-zh2d"
server_key: "49792fedbedaf27bd5626d007d693a04"
server_key_iv: "d9d6c7db74e270cf024bd590a4fbcebf"
server_key_version: "encryptionkey@0"
I20260806 22:47:24.570452 17363 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:47:24.580555 17363 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:47:24.582746 17363 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:47:24.582919 17363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:47:24.583478 17363 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:47:24.583551 17363 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:24.583598 17363 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:47:24.583628 17363 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 22:47:24 dist-test-slave-zh2d krb5kdc[16793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056444, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.40.67@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM
I20260806 22:47:24.591621 17363 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.67:45929
I20260806 22:47:24.591709 17538 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.67:45929 every 8 connection(s)
I20260806 22:47:24.592051 17363 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
I20260806 22:47:24.595089 16931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:24.584130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.67:54579 (local address 127.6.40.126:39333)
0806 22:47:24.584170 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:24.584172 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:24.584428 (+ 256us) server_negotiation.cc:407] Connection header received
0806 22:47:24.585052 (+ 624us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:24.585054 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:24.585068 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:24.585121 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:24.585826 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:24.586648 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:24.588414 (+ 1766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:24.588758 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:24.588886 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:24.591935 (+ 3049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:24.591937 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:24.591937 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:24.591946 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:24.593589 (+ 1643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:24.594081 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:24.594082 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:24.594082 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:24.594146 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:24.594410 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:24.594411 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:24.594412 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:24.594539 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:24.594615 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:24.594902 (+ 287us) server_negotiation.cc:299] Negotiation successful
0806 22:47:24.594996 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 22:47:24.595296 17392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:24.584121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:39333 (local address 127.6.40.67:54579)
0806 22:47:24.584297 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:24.584339 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:24.584936 (+ 597us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:24.585201 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:24.585204 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:24.585517 (+ 313us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:24.585717 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:24.585722 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:24.586783 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:24.586785 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:24.588315 (+ 1530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:24.588318 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:24.588931 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:24.588932 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:24.589117 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:24.589696 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:24.589707 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:24.591808 (+ 2101us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:47:24.593705 (+ 1897us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:24.593733 (+ 28us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:24.593735 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:24.593980 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:24.594217 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:24.594218 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:24.594219 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:24.594339 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:24.594617 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:47:24.594625 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:47:24.594768 (+ 143us) client_negotiation.cc:769] Sending connection context
0806 22:47:24.594880 (+ 112us) client_negotiation.cc:240] Negotiation successful
0806 22:47:24.595049 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":94,"threads_started":1}
I20260806 22:47:24.595871 17539 heartbeater.cc:347] Connected to a master server at 127.6.40.126:39333
I20260806 22:47:24.595965 17539 heartbeater.cc:464] Registering TS with master...
I20260806 22:47:24.596165 17539 heartbeater.cc:511] Master 127.6.40.126:39333 requested a full tablet report, sending...
I20260806 22:47:24.596673 16857 ts_manager.cc:194] Registered new tserver with Master: 4091dc731c9a4e15b38508539576d38e (127.6.40.67:45929)
I20260806 22:47:24.597162 16857 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.40.67@KRBTEST.COM'} at 127.6.40.67:54579
I20260806 22:47:24.600735 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 17363
I20260806 22:47:24.600894 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/minicluster-data/ts-2/wal/instance
I20260806 22:47:24.601109 6305 external_mini_cluster.cc:1428] Setting key 635305c794f0d851ff48472a5743102e
I20260806 22:47:24.602162 6305 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:47:24.603005 16931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:24.597115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.67:41311 (local address 127.6.40.126:39333)
0806 22:47:24.597158 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:24.597160 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:24.597236 (+ 76us) server_negotiation.cc:407] Connection header received
0806 22:47:24.597305 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:24.597306 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:24.597316 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:24.597366 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:24.597703 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:24.598185 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:24.598890 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:24.599096 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:24.599173 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:24.600088 (+ 915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:24.600089 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:24.600089 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:24.600096 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:24.601712 (+ 1616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:24.602129 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:24.602130 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:24.602131 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:24.602195 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:24.602511 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:24.602512 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:24.602512 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:24.602634 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:24.602685 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:24.602885 (+ 200us) server_negotiation.cc:299] Negotiation successful
0806 22:47:24.602946 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 22:47:24.603130 17549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:24.597050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:39333 (local address 127.6.40.67:41311)
0806 22:47:24.597179 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:24.597190 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:24.597250 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:24.597368 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:24.597368 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:24.597558 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:24.597625 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:24.597627 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:24.598290 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:24.598291 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:24.598805 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:24.598806 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:24.599172 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:24.599173 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:24.599243 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:24.599690 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:24.599695 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:24.599977 (+ 282us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:47:24.601812 (+ 1835us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:24.601813 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:24.601813 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:24.602041 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:24.602280 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:24.602281 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:24.602281 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:24.602384 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:24.602701 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:47:24.602702 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:47:24.602757 (+ 55us) client_negotiation.cc:769] Sending connection context
0806 22:47:24.602836 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 22:47:24.602977 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"mutex_wait_us":2,"thread_start_us":74,"threads_started":1}
Aug 06 22:47:24 dist-test-slave-zh2d krb5kdc[16793](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056444, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 22:47:24 dist-test-slave-zh2d krb5kdc[16793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:47:24 dist-test-slave-zh2d krb5kdc[16793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
W20260806 22:47:24.624737 16931 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:47:24.619310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59684 (local address 127.6.40.126:39333)
0806 22:47:24.619376 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:24.619378 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:24.619387 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:47:24.619412 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:24.619413 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:24.619422 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:24.619473 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:24.620046 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:24.620559 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:24.621434 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:24.621620 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:24.621675 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:24.624663 (+ 2988us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59684: BlockingRecv error: recv got EOF from 127.0.0.1:59684 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Aug 06 22:47:24 dist-test-slave-zh2d krb5kdc[16793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056444, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM
I20260806 22:47:24.632934 16931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:24.625715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59688 (local address 127.6.40.126:39333)
0806 22:47:24.625781 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:24.625784 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:24.625869 (+ 85us) server_negotiation.cc:407] Connection header received
0806 22:47:24.625940 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:24.625940 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:24.625950 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:24.625994 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:24.626492 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:24.627061 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:24.627735 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:24.627977 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:24.628034 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:24.630266 (+ 2232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:24.630268 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:24.630268 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:24.630275 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:24.631750 (+ 1475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:24.632110 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:24.632111 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:24.632111 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:24.632190 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:24.632462 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:24.632463 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:24.632463 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:24.632575 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:24.632660 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:24.632746 (+ 86us) server_negotiation.cc:299] Negotiation successful
0806 22:47:24.632801 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 22:47:24.634999 16857 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59688:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:47:24.636755 16857 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:47:24.644026 17570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:24.639419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.66:44463 (local address 127.0.0.1:57432)
0806 22:47:24.639887 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:24.639898 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:24.639969 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:24.640217 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:24.640221 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:24.640224 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:47:24.640312 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:24.640314 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:24.641785 (+ 1471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:24.641786 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:24.642805 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:24.642807 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:24.643715 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:24.643716 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:24.643799 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:24.643810 (+ 11us) client_negotiation.cc:769] Sending connection context
0806 22:47:24.643887 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 22:47:24.643933 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":445,"mutex_wait_us":3,"thread_start_us":88,"threads_started":1}
I20260806 22:47:24.644030 17568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:24.639382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.65:38371 (local address 127.0.0.1:35748)
0806 22:47:24.639527 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:24.639572 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:24.639650 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:24.640285 (+ 635us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:24.640286 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:24.640288 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:47:24.640364 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:24.640366 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:24.641411 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:24.641413 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:24.642624 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:24.642627 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:24.643693 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:24.643694 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:24.643790 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:24.643806 (+ 16us) client_negotiation.cc:769] Sending connection context
0806 22:47:24.643867 (+ 61us) client_negotiation.cc:240] Negotiation successful
0806 22:47:24.643929 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"mutex_wait_us":3,"thread_start_us":97,"threads_started":1}
I20260806 22:47:24.644375 17571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:24.639644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35748 (local address 127.6.40.65:38371)
0806 22:47:24.640095 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:24.640100 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:24.640112 (+ 12us) server_negotiation.cc:407] Connection header received
0806 22:47:24.640157 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:24.640159 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:24.640177 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:24.640441 (+ 264us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:47:24.640509 (+ 68us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:24.641308 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:24.642727 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:24.643569 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:24.644080 (+ 511us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:24.644185 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:24.644216 (+ 31us) server_negotiation.cc:299] Negotiation successful
0806 22:47:24.644265 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":110,"threads_started":1}
I20260806 22:47:24.644944 17569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:24.639636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.67:45929 (local address 127.0.0.1:37116)
0806 22:47:24.640140 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:24.640151 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:24.640208 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:24.640474 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:24.640475 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:24.640477 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:47:24.640514 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:24.640515 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:24.642880 (+ 2365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:24.642881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:24.643529 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:24.643530 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:24.644660 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:24.644661 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:24.644726 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:24.644730 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:47:24.644780 (+ 50us) client_negotiation.cc:240] Negotiation successful
0806 22:47:24.644821 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":65,"threads_started":1}
I20260806 22:47:24.644995 17574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:24.639838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37116 (local address 127.6.40.67:45929)
0806 22:47:24.640072 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:24.640077 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:24.640267 (+ 190us) server_negotiation.cc:407] Connection header received
0806 22:47:24.640310 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:24.640312 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:24.640328 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:24.640402 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:47:24.640604 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:24.641692 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:24.643619 (+ 1927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:24.644456 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:24.644531 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:24.644628 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:24.644810 (+ 182us) server_negotiation.cc:299] Negotiation successful
0806 22:47:24.644891 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":124,"threads_started":1}
I20260806 22:47:24.645193 17052 tablet_service.cc:1467] Processing CreateTablet for tablet f5e8a8b826a5422b82601e62c765a7f9 (DEFAULT_TABLE table=test-table [id=8452d5bbce154920bb3c01ea36fb89dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:47:24.645617 17052 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5e8a8b826a5422b82601e62c765a7f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:47:24.645684 17450 tablet_service.cc:1467] Processing CreateTablet for tablet f5e8a8b826a5422b82601e62c765a7f9 (DEFAULT_TABLE table=test-table [id=8452d5bbce154920bb3c01ea36fb89dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:47:24.646039 17450 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5e8a8b826a5422b82601e62c765a7f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:47:24.649667 17576 tablet_bootstrap.cc:492] T f5e8a8b826a5422b82601e62c765a7f9 P 4091dc731c9a4e15b38508539576d38e: Bootstrap starting.
I20260806 22:47:24.644060 17573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:24.639655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57432 (local address 127.6.40.66:44463)
0806 22:47:24.639869 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:24.639874 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:24.639956 (+ 82us) server_negotiation.cc:407] Connection header received
0806 22:47:24.640036 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:24.640038 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:24.640057 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:24.640407 (+ 350us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:47:24.640479 (+ 72us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:24.641685 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:24.642912 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:24.643636 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:24.643707 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:24.643790 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:24.643912 (+ 122us) server_negotiation.cc:299] Negotiation successful
0806 22:47:24.643952 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":120,"threads_started":1}
I20260806 22:47:24.650530 17577 tablet_bootstrap.cc:492] T f5e8a8b826a5422b82601e62c765a7f9 P 1a3c36e213304396bf7ebddea9300695: Bootstrap starting.
I20260806 22:47:24.650588 17576 tablet_bootstrap.cc:654] T f5e8a8b826a5422b82601e62c765a7f9 P 4091dc731c9a4e15b38508539576d38e: Neither blocks nor log segments found. Creating new log.
I20260806 22:47:24.650795 17257 tablet_service.cc:1467] Processing CreateTablet for tablet f5e8a8b826a5422b82601e62c765a7f9 (DEFAULT_TABLE table=test-table [id=8452d5bbce154920bb3c01ea36fb89dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:47:24.651000 17576 log.cc:824] T f5e8a8b826a5422b82601e62c765a7f9 P 4091dc731c9a4e15b38508539576d38e: Log is configured to *not* fsync() on all Append() calls
I20260806 22:47:24.651154 17257 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5e8a8b826a5422b82601e62c765a7f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:47:24.651203 17577 tablet_bootstrap.cc:654] T f5e8a8b826a5422b82601e62c765a7f9 P 1a3c36e213304396bf7ebddea9300695: Neither blocks nor log segments found. Creating new log.
I20260806 22:47:24.651535 17577 log.cc:824] T f5e8a8b826a5422b82601e62c765a7f9 P 1a3c36e213304396bf7ebddea9300695: Log is configured to *not* fsync() on all Append() calls
I20260806 22:47:24.652385 17577 tablet_bootstrap.cc:492] T f5e8a8b826a5422b82601e62c765a7f9 P 1a3c36e213304396bf7ebddea9300695: No bootstrap required, opened a new log
I20260806 22:47:24.652464 17577 ts_tablet_manager.cc:1397] T f5e8a8b826a5422b82601e62c765a7f9 P 1a3c36e213304396bf7ebddea9300695: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:47:24.654745 17577 raft_consensus.cc:348] T f5e8a8b826a5422b82601e62c765a7f9 P 1a3c36e213304396bf7ebddea9300695 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a3c36e213304396bf7ebddea9300695" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 38371 } } peers { permanent_uuid: "a712a3a80999468ea0a596422fb8f3a1" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 44463 } } peers { permanent_uuid: "4091dc731c9a4e15b38508539576d38e" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45929 } }
I20260806 22:47:24.654955 17577 raft_consensus.cc:374] T f5e8a8b826a5422b82601e62c765a7f9 P 1a3c36e213304396bf7ebddea9300695 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:47:24.654997 17577 raft_consensus.cc:729] T f5e8a8b826a5422b82601e62c765a7f9 P 1a3c36e213304396bf7ebddea9300695 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a3c36e213304396bf7ebddea9300695, State: Initialized, Role: FOLLOWER
I20260806 22:47:24.655138 17577 consensus_queue.cc:260] T f5e8a8b826a5422b82601e62c765a7f9 P 1a3c36e213304396bf7ebddea9300695 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a3c36e213304396bf7ebddea9300695" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 38371 } } peers { permanent_uuid: "a712a3a80999468ea0a596422fb8f3a1" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 44463 } } peers { permanent_uuid: "4091dc731c9a4e15b38508539576d38e" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45929 } }
I20260806 22:47:24.655434 17577 ts_tablet_manager.cc:1428] T f5e8a8b826a5422b82601e62c765a7f9 P 1a3c36e213304396bf7ebddea9300695: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:47:24.655661 17154 heartbeater.cc:503] Master 127.6.40.126:39333 was elected leader, sending a full tablet report...
I20260806 22:47:24.655781 17581 tablet_bootstrap.cc:492] T f5e8a8b826a5422b82601e62c765a7f9 P a712a3a80999468ea0a596422fb8f3a1: Bootstrap starting.
I20260806 22:47:24.655838 17576 tablet_bootstrap.cc:492] T f5e8a8b826a5422b82601e62c765a7f9 P 4091dc731c9a4e15b38508539576d38e: No bootstrap required, opened a new log
I20260806 22:47:24.655933 17576 ts_tablet_manager.cc:1397] T f5e8a8b826a5422b82601e62c765a7f9 P 4091dc731c9a4e15b38508539576d38e: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260806 22:47:24.656539 17581 tablet_bootstrap.cc:654] T f5e8a8b826a5422b82601e62c765a7f9 P a712a3a80999468ea0a596422fb8f3a1: Neither blocks nor log segments found. Creating new log.
I20260806 22:47:24.656955 17581 log.cc:824] T f5e8a8b826a5422b82601e62c765a7f9 P a712a3a80999468ea0a596422fb8f3a1: Log is configured to *not* fsync() on all Append() calls
I20260806 22:47:24.657663 17581 tablet_bootstrap.cc:492] T f5e8a8b826a5422b82601e62c765a7f9 P a712a3a80999468ea0a596422fb8f3a1: No bootstrap required, opened a new log
I20260806 22:47:24.657748 17581 ts_tablet_manager.cc:1397] T f5e8a8b826a5422b82601e62c765a7f9 P a712a3a80999468ea0a596422fb8f3a1: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:47:24.657869 17576 raft_consensus.cc:348] T f5e8a8b826a5422b82601e62c765a7f9 P 4091dc731c9a4e15b38508539576d38e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a3c36e213304396bf7ebddea9300695" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 38371 } } peers { permanent_uuid: "a712a3a80999468ea0a596422fb8f3a1" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 44463 } } peers { permanent_uuid: "4091dc731c9a4e15b38508539576d38e" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45929 } }
I20260806 22:47:24.658004 17576 raft_consensus.cc:374] T f5e8a8b826a5422b82601e62c765a7f9 P 4091dc731c9a4e15b38508539576d38e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:47:24.658026 17576 raft_consensus.cc:729] T f5e8a8b826a5422b82601e62c765a7f9 P 4091dc731c9a4e15b38508539576d38e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4091dc731c9a4e15b38508539576d38e, State: Initialized, Role: FOLLOWER
I20260806 22:47:24.658146 17576 consensus_queue.cc:260] T f5e8a8b826a5422b82601e62c765a7f9 P 4091dc731c9a4e15b38508539576d38e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a3c36e213304396bf7ebddea9300695" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 38371 } } peers { permanent_uuid: "a712a3a80999468ea0a596422fb8f3a1" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 44463 } } peers { permanent_uuid: "4091dc731c9a4e15b38508539576d38e" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45929 } }
I20260806 22:47:24.658401 17576 ts_tablet_manager.cc:1428] T f5e8a8b826a5422b82601e62c765a7f9 P 4091dc731c9a4e15b38508539576d38e: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260806 22:47:24.658459 17539 heartbeater.cc:503] Master 127.6.40.126:39333 was elected leader, sending a full tablet report...
I20260806 22:47:24.659689 17581 raft_consensus.cc:348] T f5e8a8b826a5422b82601e62c765a7f9 P a712a3a80999468ea0a596422fb8f3a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a3c36e213304396bf7ebddea9300695" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 38371 } } peers { permanent_uuid: "a712a3a80999468ea0a596422fb8f3a1" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 44463 } } peers { permanent_uuid: "4091dc731c9a4e15b38508539576d38e" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45929 } }
I20260806 22:47:24.659847 17581 raft_consensus.cc:374] T f5e8a8b826a5422b82601e62c765a7f9 P a712a3a80999468ea0a596422fb8f3a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:47:24.659871 17581 raft_consensus.cc:729] T f5e8a8b826a5422b82601e62c765a7f9 P a712a3a80999468ea0a596422fb8f3a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a712a3a80999468ea0a596422fb8f3a1, State: Initialized, Role: FOLLOWER
I20260806 22:47:24.659979 17581 consensus_queue.cc:260] T f5e8a8b826a5422b82601e62c765a7f9 P a712a3a80999468ea0a596422fb8f3a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a3c36e213304396bf7ebddea9300695" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 38371 } } peers { permanent_uuid: "a712a3a80999468ea0a596422fb8f3a1" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 44463 } } peers { permanent_uuid: "4091dc731c9a4e15b38508539576d38e" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45929 } }
I20260806 22:47:24.660207 17581 ts_tablet_manager.cc:1428] T f5e8a8b826a5422b82601e62c765a7f9 P a712a3a80999468ea0a596422fb8f3a1: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 22:47:24.660310 17354 heartbeater.cc:503] Master 127.6.40.126:39333 was elected leader, sending a full tablet report...
W20260806 22:47:24.676156 17355 tablet.cc:2369] T f5e8a8b826a5422b82601e62c765a7f9 P a712a3a80999468ea0a596422fb8f3a1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:47:24.753338 17156 tablet.cc:2369] T f5e8a8b826a5422b82601e62c765a7f9 P 1a3c36e213304396bf7ebddea9300695: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:47:24.842751 17540 tablet.cc:2369] T f5e8a8b826a5422b82601e62c765a7f9 P 4091dc731c9a4e15b38508539576d38e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:47:24.933159 17580 raft_consensus.cc:482] T f5e8a8b826a5422b82601e62c765a7f9 P 1a3c36e213304396bf7ebddea9300695 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:47:24.933405 17580 raft_consensus.cc:504] T f5e8a8b826a5422b82601e62c765a7f9 P 1a3c36e213304396bf7ebddea9300695 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a3c36e213304396bf7ebddea9300695" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 38371 } } peers { permanent_uuid: "a712a3a80999468ea0a596422fb8f3a1" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 44463 } } peers { permanent_uuid: "4091dc731c9a4e15b38508539576d38e" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45929 } }
I20260806 22:47:24.933813 17580 leader_election.cc:288] T f5e8a8b826a5422b82601e62c765a7f9 P 1a3c36e213304396bf7ebddea9300695 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a712a3a80999468ea0a596422fb8f3a1 (127.6.40.66:44463), 4091dc731c9a4e15b38508539576d38e (127.6.40.67:45929)
I20260806 22:47:24.937070 17586 raft_consensus.cc:482] T f5e8a8b826a5422b82601e62c765a7f9 P 4091dc731c9a4e15b38508539576d38e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:47:24.937206 17586 raft_consensus.cc:504] T f5e8a8b826a5422b82601e62c765a7f9 P 4091dc731c9a4e15b38508539576d38e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a3c36e213304396bf7ebddea9300695" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 38371 } } peers { permanent_uuid: "a712a3a80999468ea0a596422fb8f3a1" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 44463 } } peers { permanent_uuid: "4091dc731c9a4e15b38508539576d38e" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45929 } }
I20260806 22:47:24.937269 17592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:24.934387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.67:45929 (local address 127.6.40.65:56007)
0806 22:47:24.934495 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:24.934506 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:24.934569 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:24.934749 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:24.934749 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:24.934752 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:47:24.934860 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:24.934861 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:24.935527 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:24.935528 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:24.936353 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:24.936356 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:24.936966 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:24.936967 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:24.937046 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:24.937052 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 22:47:24.937112 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 22:47:24.937165 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91,"mutex_wait_us":21,"thread_start_us":62,"threads_started":1}
I20260806 22:47:24.937345 17573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:24.934167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.65:54935 (local address 127.6.40.66:44463)
0806 22:47:24.934262 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:24.934265 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:24.934397 (+ 132us) server_negotiation.cc:407] Connection header received
0806 22:47:24.934543 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:24.934544 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:24.934564 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:24.934645 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:47:24.934839 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:24.935384 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:24.936490 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:24.936913 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:24.936984 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:24.936996 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:24.937200 (+ 204us) server_negotiation.cc:299] Negotiation successful
0806 22:47:24.937239 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 22:47:24.937554 17591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:24.934079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.66:44463 (local address 127.6.40.65:54935)
0806 22:47:24.934265 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:24.934277 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:24.934420 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:24.934659 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:24.934660 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:24.934663 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:47:24.934752 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:24.934754 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:24.935477 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:24.935479 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:24.936369 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:24.936372 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:24.937027 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:24.937029 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:24.937108 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:24.937111 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:47:24.937429 (+ 318us) client_negotiation.cc:240] Negotiation successful
0806 22:47:24.937479 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"mutex_wait_us":31,"thread_start_us":121,"threads_started":1}
I20260806 22:47:24.937561 17574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:24.934519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.65:56007 (local address 127.6.40.67:45929)
0806 22:47:24.934565 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:24.934567 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:24.934576 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:47:24.934652 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:24.934653 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:24.934664 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:24.934722 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:47:24.934941 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:24.935411 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:24.936450 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:24.936817 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:24.936962 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:24.936975 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:24.937160 (+ 185us) server_negotiation.cc:299] Negotiation successful
0806 22:47:24.937198 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 22:47:24.937593 17586 leader_election.cc:288] T f5e8a8b826a5422b82601e62c765a7f9 P 4091dc731c9a4e15b38508539576d38e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1a3c36e213304396bf7ebddea9300695 (127.6.40.65:38371), a712a3a80999468ea0a596422fb8f3a1 (127.6.40.66:44463)
I20260806 22:47:24.937927 17286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5e8a8b826a5422b82601e62c765a7f9" candidate_uuid: "1a3c36e213304396bf7ebddea9300695" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a712a3a80999468ea0a596422fb8f3a1" is_pre_election: true
I20260806 22:47:24.938097 17286 raft_consensus.cc:2454] T f5e8a8b826a5422b82601e62c765a7f9 P a712a3a80999468ea0a596422fb8f3a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a3c36e213304396bf7ebddea9300695 in term 0.
I20260806 22:47:24.938124 17476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5e8a8b826a5422b82601e62c765a7f9" candidate_uuid: "1a3c36e213304396bf7ebddea9300695" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4091dc731c9a4e15b38508539576d38e" is_pre_election: true
I20260806 22:47:24.938215 17476 raft_consensus.cc:2454] T f5e8a8b826a5422b82601e62c765a7f9 P 4091dc731c9a4e15b38508539576d38e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a3c36e213304396bf7ebddea9300695 in term 0.
I20260806 22:47:24.938555 16984 leader_election.cc:302] T f5e8a8b826a5422b82601e62c765a7f9 P 1a3c36e213304396bf7ebddea9300695 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a3c36e213304396bf7ebddea9300695, 4091dc731c9a4e15b38508539576d38e; no voters:
I20260806 22:47:24.938724 17580 raft_consensus.cc:2781] T f5e8a8b826a5422b82601e62c765a7f9 P 1a3c36e213304396bf7ebddea9300695 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:47:24.938805 17580 raft_consensus.cc:482] T f5e8a8b826a5422b82601e62c765a7f9 P 1a3c36e213304396bf7ebddea9300695 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:47:24.938829 17580 raft_consensus.cc:3037] T f5e8a8b826a5422b82601e62c765a7f9 P 1a3c36e213304396bf7ebddea9300695 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:47:24.939733 17580 raft_consensus.cc:504] T f5e8a8b826a5422b82601e62c765a7f9 P 1a3c36e213304396bf7ebddea9300695 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a3c36e213304396bf7ebddea9300695" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 38371 } } peers { permanent_uuid: "a712a3a80999468ea0a596422fb8f3a1" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 44463 } } peers { permanent_uuid: "4091dc731c9a4e15b38508539576d38e" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45929 } }
I20260806 22:47:24.939875 17580 leader_election.cc:288] T f5e8a8b826a5422b82601e62c765a7f9 P 1a3c36e213304396bf7ebddea9300695 [CANDIDATE]: Term 1 election: Requested vote from peers a712a3a80999468ea0a596422fb8f3a1 (127.6.40.66:44463), 4091dc731c9a4e15b38508539576d38e (127.6.40.67:45929)
I20260806 22:47:24.940078 17476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5e8a8b826a5422b82601e62c765a7f9" candidate_uuid: "1a3c36e213304396bf7ebddea9300695" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4091dc731c9a4e15b38508539576d38e"
I20260806 22:47:24.940079 17286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5e8a8b826a5422b82601e62c765a7f9" candidate_uuid: "1a3c36e213304396bf7ebddea9300695" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a712a3a80999468ea0a596422fb8f3a1"
I20260806 22:47:24.940148 17476 raft_consensus.cc:3037] T f5e8a8b826a5422b82601e62c765a7f9 P 4091dc731c9a4e15b38508539576d38e [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:47:24.940153 17286 raft_consensus.cc:3037] T f5e8a8b826a5422b82601e62c765a7f9 P a712a3a80999468ea0a596422fb8f3a1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:47:24.941139 17286 raft_consensus.cc:2454] T f5e8a8b826a5422b82601e62c765a7f9 P a712a3a80999468ea0a596422fb8f3a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a3c36e213304396bf7ebddea9300695 in term 1.
I20260806 22:47:24.941265 17476 raft_consensus.cc:2454] T f5e8a8b826a5422b82601e62c765a7f9 P 4091dc731c9a4e15b38508539576d38e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a3c36e213304396bf7ebddea9300695 in term 1.
I20260806 22:47:24.941331 16984 leader_election.cc:302] T f5e8a8b826a5422b82601e62c765a7f9 P 1a3c36e213304396bf7ebddea9300695 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a3c36e213304396bf7ebddea9300695, a712a3a80999468ea0a596422fb8f3a1; no voters:
I20260806 22:47:24.941479 17580 raft_consensus.cc:2781] T f5e8a8b826a5422b82601e62c765a7f9 P 1a3c36e213304396bf7ebddea9300695 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:47:24.941607 17392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:24.937802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.65:38371 (local address 127.6.40.67:55057)
0806 22:47:24.937887 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:24.937897 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:24.937960 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:24.938382 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:24.938384 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:24.938386 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:47:24.938444 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:24.938445 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:24.939323 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:24.939324 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:24.940560 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:24.940563 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:24.941336 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:24.941338 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:24.941423 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:24.941427 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:47:24.941480 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 22:47:24.941525 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65}
I20260806 22:47:24.941632 17580 raft_consensus.cc:686] T f5e8a8b826a5422b82601e62c765a7f9 P 1a3c36e213304396bf7ebddea9300695 [term 1 LEADER]: Becoming Leader. State: Replica: 1a3c36e213304396bf7ebddea9300695, State: Running, Role: LEADER
I20260806 22:47:24.941754 17580 consensus_queue.cc:237] T f5e8a8b826a5422b82601e62c765a7f9 P 1a3c36e213304396bf7ebddea9300695 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a3c36e213304396bf7ebddea9300695" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 38371 } } peers { permanent_uuid: "a712a3a80999468ea0a596422fb8f3a1" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 44463 } } peers { permanent_uuid: "4091dc731c9a4e15b38508539576d38e" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45929 } }
I20260806 22:47:24.941921 17571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:24.937843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.67:55057 (local address 127.6.40.65:38371)
0806 22:47:24.937901 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:24.937903 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:24.938104 (+ 201us) server_negotiation.cc:407] Connection header received
0806 22:47:24.938129 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:24.938130 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:24.938143 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:24.938206 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:47:24.938509 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:24.939218 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:24.940673 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:24.941212 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:24.941278 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:24.941288 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:24.941805 (+ 517us) server_negotiation.cc:299] Negotiation successful
0806 22:47:24.941854 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 22:47:24.942001 17573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:24.937948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.67:34403 (local address 127.6.40.66:44463)
0806 22:47:24.938314 (+ 366us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:24.938316 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:24.938378 (+ 62us) server_negotiation.cc:407] Connection header received
0806 22:47:24.938442 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:24.938443 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:24.938452 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:24.938506 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:47:24.938805 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:24.939351 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:24.941061 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:24.941565 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:24.941608 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:24.941618 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:24.941895 (+ 277us) server_negotiation.cc:299] Negotiation successful
0806 22:47:24.941934 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343}
I20260806 22:47:24.942435 17593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:24.937808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.66:44463 (local address 127.6.40.67:34403)
0806 22:47:24.938322 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:24.938332 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:24.938391 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:24.938598 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:24.938599 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:24.938600 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:47:24.938688 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:24.938690 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:24.939458 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:24.939459 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:24.940958 (+ 1499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:24.940960 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:24.941704 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:24.941705 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:24.941787 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:24.941804 (+ 17us) client_negotiation.cc:769] Sending connection context
0806 22:47:24.942113 (+ 309us) client_negotiation.cc:240] Negotiation successful
0806 22:47:24.942165 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"mutex_wait_us":145,"thread_start_us":96,"threads_started":1}
I20260806 22:47:24.942482 17057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5e8a8b826a5422b82601e62c765a7f9" candidate_uuid: "4091dc731c9a4e15b38508539576d38e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a3c36e213304396bf7ebddea9300695" is_pre_election: true
I20260806 22:47:24.942612 16857 catalog_manager.cc:5495] T f5e8a8b826a5422b82601e62c765a7f9 P 1a3c36e213304396bf7ebddea9300695 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1a3c36e213304396bf7ebddea9300695 (127.6.40.65). New cstate: current_term: 1 leader_uuid: "1a3c36e213304396bf7ebddea9300695" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a3c36e213304396bf7ebddea9300695" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 38371 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a712a3a80999468ea0a596422fb8f3a1" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 44463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4091dc731c9a4e15b38508539576d38e" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45929 } health_report { overall_health: UNKNOWN } } }
I20260806 22:47:24.943154 17286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5e8a8b826a5422b82601e62c765a7f9" candidate_uuid: "4091dc731c9a4e15b38508539576d38e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a712a3a80999468ea0a596422fb8f3a1" is_pre_election: true
I20260806 22:47:24.943256 17286 raft_consensus.cc:2379] T f5e8a8b826a5422b82601e62c765a7f9 P a712a3a80999468ea0a596422fb8f3a1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4091dc731c9a4e15b38508539576d38e in current term 1: Already voted for candidate 1a3c36e213304396bf7ebddea9300695 in this term.
I20260806 22:47:24.943503 17385 leader_election.cc:302] T f5e8a8b826a5422b82601e62c765a7f9 P 4091dc731c9a4e15b38508539576d38e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4091dc731c9a4e15b38508539576d38e; no voters: 1a3c36e213304396bf7ebddea9300695, a712a3a80999468ea0a596422fb8f3a1
I20260806 22:47:24.943652 17586 raft_consensus.cc:2726] T f5e8a8b826a5422b82601e62c765a7f9 P 4091dc731c9a4e15b38508539576d38e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 22:47:24.997598 17571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:24.994784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35762 (local address 127.6.40.65:38371)
0806 22:47:24.994852 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:24.994855 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:24.994864 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:47:24.994912 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:24.994913 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:24.994933 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:24.995024 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:47:24.995174 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:24.995791 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:24.996640 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:24.996924 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:24.997004 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:24.997218 (+ 214us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:47:24.997342 (+ 124us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:47:24.997441 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:24.997484 (+ 43us) server_negotiation.cc:299] Negotiation successful
0806 22:47:24.997508 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 22:47:24.999771 17476 raft_consensus.cc:1261] T f5e8a8b826a5422b82601e62c765a7f9 P 4091dc731c9a4e15b38508539576d38e [term 1 FOLLOWER]: Refusing update from remote peer 1a3c36e213304396bf7ebddea9300695: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:47:25.000093 17286 raft_consensus.cc:1261] T f5e8a8b826a5422b82601e62c765a7f9 P a712a3a80999468ea0a596422fb8f3a1 [term 1 FOLLOWER]: Refusing update from remote peer 1a3c36e213304396bf7ebddea9300695: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:47:25.000185 17580 consensus_queue.cc:1035] T f5e8a8b826a5422b82601e62c765a7f9 P 1a3c36e213304396bf7ebddea9300695 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4091dc731c9a4e15b38508539576d38e" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:47:25.000466 17594 consensus_queue.cc:1035] T f5e8a8b826a5422b82601e62c765a7f9 P 1a3c36e213304396bf7ebddea9300695 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a712a3a80999468ea0a596422fb8f3a1" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:47:25.003137 17598 mvcc.cc:205] Tried to move back new op lower bound from 7315687198715531264 to 7315687198481907712. Current Snapshot: MvccSnapshot[applied={T|T < 7315687198715531264}]
I20260806 22:47:25.004088 17599 mvcc.cc:205] Tried to move back new op lower bound from 7315687198715531264 to 7315687198481907712. Current Snapshot: MvccSnapshot[applied={T|T < 7315687198715531264 or (T in {7315687198715531264})}]
I20260806 22:47:25.005810 16858 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59688:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:47:25.005923 16858 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59688:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:47:25.006729 16858 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a023cd9df99d4e3db64a1b4977884980: Sending DeleteTablet for 3 replicas of tablet f5e8a8b826a5422b82601e62c765a7f9
I20260806 22:47:25.007071 17257 tablet_service.cc:1514] Processing DeleteTablet for tablet f5e8a8b826a5422b82601e62c765a7f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:47:25 UTC) from {username='oryx', principal='oryx/127.6.40.126@KRBTEST.COM'} at 127.0.0.1:57432
I20260806 22:47:25.007112 17450 tablet_service.cc:1514] Processing DeleteTablet for tablet f5e8a8b826a5422b82601e62c765a7f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:47:25 UTC) from {username='oryx', principal='oryx/127.6.40.126@KRBTEST.COM'} at 127.0.0.1:37116
I20260806 22:47:25.007140 17052 tablet_service.cc:1514] Processing DeleteTablet for tablet f5e8a8b826a5422b82601e62c765a7f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:47:25 UTC) from {username='oryx', principal='oryx/127.6.40.126@KRBTEST.COM'} at 127.0.0.1:35748
I20260806 22:47:25.007278 17609 tablet_replica.cc:330] T f5e8a8b826a5422b82601e62c765a7f9 P a712a3a80999468ea0a596422fb8f3a1: stopping tablet replica
I20260806 22:47:25.007436 17609 raft_consensus.cc:2229] T f5e8a8b826a5422b82601e62c765a7f9 P a712a3a80999468ea0a596422fb8f3a1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:47:25.007566 17609 raft_consensus.cc:2258] T f5e8a8b826a5422b82601e62c765a7f9 P a712a3a80999468ea0a596422fb8f3a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:47:25.007649 17611 tablet_replica.cc:330] T f5e8a8b826a5422b82601e62c765a7f9 P 1a3c36e213304396bf7ebddea9300695: stopping tablet replica
I20260806 22:47:25.007613 16858 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59688:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 22:47:25.007755 17611 raft_consensus.cc:2229] T f5e8a8b826a5422b82601e62c765a7f9 P 1a3c36e213304396bf7ebddea9300695 [term 1 LEADER]: Raft consensus shutting down.
W20260806 22:47:25.007771 16858 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:47:25.007890 17611 raft_consensus.cc:2258] T f5e8a8b826a5422b82601e62c765a7f9 P 1a3c36e213304396bf7ebddea9300695 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:47:25.007452 17610 tablet_replica.cc:330] T f5e8a8b826a5422b82601e62c765a7f9 P 4091dc731c9a4e15b38508539576d38e: stopping tablet replica
I20260806 22:47:25.008065 17610 raft_consensus.cc:2229] T f5e8a8b826a5422b82601e62c765a7f9 P 4091dc731c9a4e15b38508539576d38e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:47:25.008175 17610 raft_consensus.cc:2258] T f5e8a8b826a5422b82601e62c765a7f9 P 4091dc731c9a4e15b38508539576d38e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:47:25.008446 17609 ts_tablet_manager.cc:1905] T f5e8a8b826a5422b82601e62c765a7f9 P a712a3a80999468ea0a596422fb8f3a1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:47:25.008648 17611 ts_tablet_manager.cc:1905] T f5e8a8b826a5422b82601e62c765a7f9 P 1a3c36e213304396bf7ebddea9300695: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:47:25.009440 17052 tablet_service.cc:1467] Processing CreateTablet for tablet 77fd972a26eb4f96bdd109505fa77fd1 (DEFAULT_TABLE table=test-table [id=3dc395d5c14348f7a4c4a067a54f4e58]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:47:25.009509 17257 tablet_service.cc:1467] Processing CreateTablet for tablet 77fd972a26eb4f96bdd109505fa77fd1 (DEFAULT_TABLE table=test-table [id=3dc395d5c14348f7a4c4a067a54f4e58]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:47:25.009558 17052 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77fd972a26eb4f96bdd109505fa77fd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:47:25.009627 17257 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77fd972a26eb4f96bdd109505fa77fd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:47:25.010667 17611 ts_tablet_manager.cc:1918] T f5e8a8b826a5422b82601e62c765a7f9 P 1a3c36e213304396bf7ebddea9300695: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:47:25.010730 17611 log.cc:1196] T f5e8a8b826a5422b82601e62c765a7f9 P 1a3c36e213304396bf7ebddea9300695: Deleting WAL directory at /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/minicluster-data/ts-0/wal/wals/f5e8a8b826a5422b82601e62c765a7f9
I20260806 22:47:25.010900 17577 tablet_bootstrap.cc:492] T 77fd972a26eb4f96bdd109505fa77fd1 P 1a3c36e213304396bf7ebddea9300695: Bootstrap starting.
I20260806 22:47:25.011027 17611 ts_tablet_manager.cc:1939] T f5e8a8b826a5422b82601e62c765a7f9 P 1a3c36e213304396bf7ebddea9300695: Deleting consensus metadata
I20260806 22:47:25.011155 17450 tablet_service.cc:1467] Processing CreateTablet for tablet 77fd972a26eb4f96bdd109505fa77fd1 (DEFAULT_TABLE table=test-table [id=3dc395d5c14348f7a4c4a067a54f4e58]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:47:25.011260 17450 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77fd972a26eb4f96bdd109505fa77fd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:47:25.011327 17577 tablet_bootstrap.cc:654] T 77fd972a26eb4f96bdd109505fa77fd1 P 1a3c36e213304396bf7ebddea9300695: Neither blocks nor log segments found. Creating new log.
I20260806 22:47:25.011785 16834 catalog_manager.cc:4841] TS 1a3c36e213304396bf7ebddea9300695 (127.6.40.65:38371): tablet f5e8a8b826a5422b82601e62c765a7f9 (table test-table [id=8452d5bbce154920bb3c01ea36fb89dd]) successfully deleted
I20260806 22:47:25.011986 17577 tablet_bootstrap.cc:492] T 77fd972a26eb4f96bdd109505fa77fd1 P 1a3c36e213304396bf7ebddea9300695: No bootstrap required, opened a new log
I20260806 22:47:25.012030 17577 ts_tablet_manager.cc:1397] T 77fd972a26eb4f96bdd109505fa77fd1 P 1a3c36e213304396bf7ebddea9300695: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 22:47:25.012187 17577 raft_consensus.cc:348] T 77fd972a26eb4f96bdd109505fa77fd1 P 1a3c36e213304396bf7ebddea9300695 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a3c36e213304396bf7ebddea9300695" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 38371 } } peers { permanent_uuid: "a712a3a80999468ea0a596422fb8f3a1" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 44463 } } peers { permanent_uuid: "4091dc731c9a4e15b38508539576d38e" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45929 } }
I20260806 22:47:25.012248 17577 raft_consensus.cc:374] T 77fd972a26eb4f96bdd109505fa77fd1 P 1a3c36e213304396bf7ebddea9300695 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:47:25.012266 17577 raft_consensus.cc:729] T 77fd972a26eb4f96bdd109505fa77fd1 P 1a3c36e213304396bf7ebddea9300695 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a3c36e213304396bf7ebddea9300695, State: Initialized, Role: FOLLOWER
I20260806 22:47:25.012308 17577 consensus_queue.cc:260] T 77fd972a26eb4f96bdd109505fa77fd1 P 1a3c36e213304396bf7ebddea9300695 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a3c36e213304396bf7ebddea9300695" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 38371 } } peers { permanent_uuid: "a712a3a80999468ea0a596422fb8f3a1" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 44463 } } peers { permanent_uuid: "4091dc731c9a4e15b38508539576d38e" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45929 } }
I20260806 22:47:25.012553 17577 ts_tablet_manager.cc:1428] T 77fd972a26eb4f96bdd109505fa77fd1 P 1a3c36e213304396bf7ebddea9300695: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 22:47:25.013010 17576 tablet_bootstrap.cc:492] T 77fd972a26eb4f96bdd109505fa77fd1 P 4091dc731c9a4e15b38508539576d38e: Bootstrap starting.
I20260806 22:47:25.013036 17581 tablet_bootstrap.cc:492] T 77fd972a26eb4f96bdd109505fa77fd1 P a712a3a80999468ea0a596422fb8f3a1: Bootstrap starting.
I20260806 22:47:25.013269 17609 ts_tablet_manager.cc:1918] T f5e8a8b826a5422b82601e62c765a7f9 P a712a3a80999468ea0a596422fb8f3a1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:47:25.013346 17609 log.cc:1196] T f5e8a8b826a5422b82601e62c765a7f9 P a712a3a80999468ea0a596422fb8f3a1: Deleting WAL directory at /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/minicluster-data/ts-1/wal/wals/f5e8a8b826a5422b82601e62c765a7f9
I20260806 22:47:25.013490 17576 tablet_bootstrap.cc:654] T 77fd972a26eb4f96bdd109505fa77fd1 P 4091dc731c9a4e15b38508539576d38e: Neither blocks nor log segments found. Creating new log.
I20260806 22:47:25.013517 17581 tablet_bootstrap.cc:654] T 77fd972a26eb4f96bdd109505fa77fd1 P a712a3a80999468ea0a596422fb8f3a1: Neither blocks nor log segments found. Creating new log.
I20260806 22:47:25.013620 17609 ts_tablet_manager.cc:1939] T f5e8a8b826a5422b82601e62c765a7f9 P a712a3a80999468ea0a596422fb8f3a1: Deleting consensus metadata
I20260806 22:47:25.014189 17581 tablet_bootstrap.cc:492] T 77fd972a26eb4f96bdd109505fa77fd1 P a712a3a80999468ea0a596422fb8f3a1: No bootstrap required, opened a new log
I20260806 22:47:25.014247 17581 ts_tablet_manager.cc:1397] T 77fd972a26eb4f96bdd109505fa77fd1 P a712a3a80999468ea0a596422fb8f3a1: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 22:47:25.014600 17581 raft_consensus.cc:348] T 77fd972a26eb4f96bdd109505fa77fd1 P a712a3a80999468ea0a596422fb8f3a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a3c36e213304396bf7ebddea9300695" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 38371 } } peers { permanent_uuid: "a712a3a80999468ea0a596422fb8f3a1" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 44463 } } peers { permanent_uuid: "4091dc731c9a4e15b38508539576d38e" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45929 } }
I20260806 22:47:25.014688 17581 raft_consensus.cc:374] T 77fd972a26eb4f96bdd109505fa77fd1 P a712a3a80999468ea0a596422fb8f3a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:47:25.014714 17581 raft_consensus.cc:729] T 77fd972a26eb4f96bdd109505fa77fd1 P a712a3a80999468ea0a596422fb8f3a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a712a3a80999468ea0a596422fb8f3a1, State: Initialized, Role: FOLLOWER
I20260806 22:47:25.014652 17610 ts_tablet_manager.cc:1905] T f5e8a8b826a5422b82601e62c765a7f9 P 4091dc731c9a4e15b38508539576d38e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:47:25.014761 17581 consensus_queue.cc:260] T 77fd972a26eb4f96bdd109505fa77fd1 P a712a3a80999468ea0a596422fb8f3a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a3c36e213304396bf7ebddea9300695" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 38371 } } peers { permanent_uuid: "a712a3a80999468ea0a596422fb8f3a1" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 44463 } } peers { permanent_uuid: "4091dc731c9a4e15b38508539576d38e" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45929 } }
I20260806 22:47:25.014863 17581 ts_tablet_manager.cc:1428] T 77fd972a26eb4f96bdd109505fa77fd1 P a712a3a80999468ea0a596422fb8f3a1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 22:47:25.015098 16838 catalog_manager.cc:4841] TS a712a3a80999468ea0a596422fb8f3a1 (127.6.40.66:44463): tablet f5e8a8b826a5422b82601e62c765a7f9 (table test-table [id=8452d5bbce154920bb3c01ea36fb89dd]) successfully deleted
I20260806 22:47:25.015799 17576 tablet_bootstrap.cc:492] T 77fd972a26eb4f96bdd109505fa77fd1 P 4091dc731c9a4e15b38508539576d38e: No bootstrap required, opened a new log
I20260806 22:47:25.015856 17576 ts_tablet_manager.cc:1397] T 77fd972a26eb4f96bdd109505fa77fd1 P 4091dc731c9a4e15b38508539576d38e: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260806 22:47:25.016033 17576 raft_consensus.cc:348] T 77fd972a26eb4f96bdd109505fa77fd1 P 4091dc731c9a4e15b38508539576d38e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a3c36e213304396bf7ebddea9300695" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 38371 } } peers { permanent_uuid: "a712a3a80999468ea0a596422fb8f3a1" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 44463 } } peers { permanent_uuid: "4091dc731c9a4e15b38508539576d38e" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45929 } }
I20260806 22:47:25.016127 17576 raft_consensus.cc:374] T 77fd972a26eb4f96bdd109505fa77fd1 P 4091dc731c9a4e15b38508539576d38e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:47:25.016155 17576 raft_consensus.cc:729] T 77fd972a26eb4f96bdd109505fa77fd1 P 4091dc731c9a4e15b38508539576d38e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4091dc731c9a4e15b38508539576d38e, State: Initialized, Role: FOLLOWER
I20260806 22:47:25.016199 17576 consensus_queue.cc:260] T 77fd972a26eb4f96bdd109505fa77fd1 P 4091dc731c9a4e15b38508539576d38e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a3c36e213304396bf7ebddea9300695" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 38371 } } peers { permanent_uuid: "a712a3a80999468ea0a596422fb8f3a1" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 44463 } } peers { permanent_uuid: "4091dc731c9a4e15b38508539576d38e" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45929 } }
I20260806 22:47:25.016302 17576 ts_tablet_manager.cc:1428] T 77fd972a26eb4f96bdd109505fa77fd1 P 4091dc731c9a4e15b38508539576d38e: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260806 22:47:25.016799 17610 ts_tablet_manager.cc:1918] T f5e8a8b826a5422b82601e62c765a7f9 P 4091dc731c9a4e15b38508539576d38e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:47:25.016932 17610 log.cc:1196] T f5e8a8b826a5422b82601e62c765a7f9 P 4091dc731c9a4e15b38508539576d38e: Deleting WAL directory at /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/minicluster-data/ts-2/wal/wals/f5e8a8b826a5422b82601e62c765a7f9
I20260806 22:47:25.017197 17610 ts_tablet_manager.cc:1939] T f5e8a8b826a5422b82601e62c765a7f9 P 4091dc731c9a4e15b38508539576d38e: Deleting consensus metadata
I20260806 22:47:25.017589 16838 catalog_manager.cc:4841] TS 4091dc731c9a4e15b38508539576d38e (127.6.40.67:45929): tablet f5e8a8b826a5422b82601e62c765a7f9 (table test-table [id=8452d5bbce154920bb3c01ea36fb89dd]) successfully deleted
I20260806 22:47:25.107563 17588 raft_consensus.cc:482] T 77fd972a26eb4f96bdd109505fa77fd1 P a712a3a80999468ea0a596422fb8f3a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:47:25.107712 17588 raft_consensus.cc:504] T 77fd972a26eb4f96bdd109505fa77fd1 P a712a3a80999468ea0a596422fb8f3a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a3c36e213304396bf7ebddea9300695" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 38371 } } peers { permanent_uuid: "a712a3a80999468ea0a596422fb8f3a1" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 44463 } } peers { permanent_uuid: "4091dc731c9a4e15b38508539576d38e" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45929 } }
I20260806 22:47:25.108131 17588 leader_election.cc:288] T 77fd972a26eb4f96bdd109505fa77fd1 P a712a3a80999468ea0a596422fb8f3a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1a3c36e213304396bf7ebddea9300695 (127.6.40.65:38371), 4091dc731c9a4e15b38508539576d38e (127.6.40.67:45929)
I20260806 22:47:25.111778 17574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:25.108524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.66:38799 (local address 127.6.40.67:45929)
0806 22:47:25.108568 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:25.108571 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:25.108633 (+ 62us) server_negotiation.cc:407] Connection header received
0806 22:47:25.108689 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:25.108690 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:25.108700 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:25.108772 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:47:25.109000 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:25.109697 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:25.110988 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:25.111348 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:25.111442 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:25.111454 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:25.111652 (+ 198us) server_negotiation.cc:299] Negotiation successful
0806 22:47:25.111703 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 22:47:25.111794 17616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:25.108302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.67:45929 (local address 127.6.40.66:38799)
0806 22:47:25.108567 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:25.108577 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:25.108635 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:25.108789 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:25.108790 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:25.108792 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:47:25.108906 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:25.108908 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:25.109786 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:25.109787 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:25.110871 (+ 1084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:25.110872 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:25.111483 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:25.111485 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:25.111582 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:25.111588 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 22:47:25.111642 (+ 54us) client_negotiation.cc:240] Negotiation successful
0806 22:47:25.111692 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":94,"threads_started":1}
I20260806 22:47:25.111876 17571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:25.108422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.66:44709 (local address 127.6.40.65:38371)
0806 22:47:25.108471 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:25.108473 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:25.108592 (+ 119us) server_negotiation.cc:407] Connection header received
0806 22:47:25.108733 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:25.108734 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:25.108766 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:25.108816 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:47:25.109026 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:25.109503 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:25.110976 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:25.111504 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:25.111586 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:25.111598 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:25.111757 (+ 159us) server_negotiation.cc:299] Negotiation successful
0806 22:47:25.111804 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 22:47:25.112100 17617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:25.108295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.65:38371 (local address 127.6.40.66:44709)
0806 22:47:25.108516 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:25.108529 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:25.108626 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:25.108897 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:25.108898 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:25.108899 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:47:25.108961 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:25.108962 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:25.109599 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:25.109601 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:25.110848 (+ 1247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:25.110851 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:25.111595 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:25.111596 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:25.111666 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:25.111669 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:47:25.111954 (+ 285us) client_negotiation.cc:240] Negotiation successful
0806 22:47:25.112003 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":82,"threads_started":1}
I20260806 22:47:25.112073 17476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77fd972a26eb4f96bdd109505fa77fd1" candidate_uuid: "a712a3a80999468ea0a596422fb8f3a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4091dc731c9a4e15b38508539576d38e" is_pre_election: true
I20260806 22:47:25.112170 17476 raft_consensus.cc:2454] T 77fd972a26eb4f96bdd109505fa77fd1 P 4091dc731c9a4e15b38508539576d38e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a712a3a80999468ea0a596422fb8f3a1 in term 0.
I20260806 22:47:25.112350 17190 leader_election.cc:302] T 77fd972a26eb4f96bdd109505fa77fd1 P a712a3a80999468ea0a596422fb8f3a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4091dc731c9a4e15b38508539576d38e, a712a3a80999468ea0a596422fb8f3a1; no voters:
I20260806 22:47:25.112366 17057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77fd972a26eb4f96bdd109505fa77fd1" candidate_uuid: "a712a3a80999468ea0a596422fb8f3a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a3c36e213304396bf7ebddea9300695" is_pre_election: true
I20260806 22:47:25.112457 17057 raft_consensus.cc:2454] T 77fd972a26eb4f96bdd109505fa77fd1 P 1a3c36e213304396bf7ebddea9300695 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a712a3a80999468ea0a596422fb8f3a1 in term 0.
I20260806 22:47:25.112475 17588 raft_consensus.cc:2781] T 77fd972a26eb4f96bdd109505fa77fd1 P a712a3a80999468ea0a596422fb8f3a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:47:25.112505 17588 raft_consensus.cc:482] T 77fd972a26eb4f96bdd109505fa77fd1 P a712a3a80999468ea0a596422fb8f3a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:47:25.112529 17588 raft_consensus.cc:3037] T 77fd972a26eb4f96bdd109505fa77fd1 P a712a3a80999468ea0a596422fb8f3a1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:47:25.113372 17588 raft_consensus.cc:504] T 77fd972a26eb4f96bdd109505fa77fd1 P a712a3a80999468ea0a596422fb8f3a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a3c36e213304396bf7ebddea9300695" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 38371 } } peers { permanent_uuid: "a712a3a80999468ea0a596422fb8f3a1" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 44463 } } peers { permanent_uuid: "4091dc731c9a4e15b38508539576d38e" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45929 } }
I20260806 22:47:25.113520 17588 leader_election.cc:288] T 77fd972a26eb4f96bdd109505fa77fd1 P a712a3a80999468ea0a596422fb8f3a1 [CANDIDATE]: Term 1 election: Requested vote from peers 1a3c36e213304396bf7ebddea9300695 (127.6.40.65:38371), 4091dc731c9a4e15b38508539576d38e (127.6.40.67:45929)
I20260806 22:47:25.113660 17057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77fd972a26eb4f96bdd109505fa77fd1" candidate_uuid: "a712a3a80999468ea0a596422fb8f3a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a3c36e213304396bf7ebddea9300695"
I20260806 22:47:25.113756 17057 raft_consensus.cc:3037] T 77fd972a26eb4f96bdd109505fa77fd1 P 1a3c36e213304396bf7ebddea9300695 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:47:25.113749 17476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77fd972a26eb4f96bdd109505fa77fd1" candidate_uuid: "a712a3a80999468ea0a596422fb8f3a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4091dc731c9a4e15b38508539576d38e"
I20260806 22:47:25.113829 17476 raft_consensus.cc:3037] T 77fd972a26eb4f96bdd109505fa77fd1 P 4091dc731c9a4e15b38508539576d38e [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:47:25.114643 17476 raft_consensus.cc:2454] T 77fd972a26eb4f96bdd109505fa77fd1 P 4091dc731c9a4e15b38508539576d38e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a712a3a80999468ea0a596422fb8f3a1 in term 1.
I20260806 22:47:25.114643 17057 raft_consensus.cc:2454] T 77fd972a26eb4f96bdd109505fa77fd1 P 1a3c36e213304396bf7ebddea9300695 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a712a3a80999468ea0a596422fb8f3a1 in term 1.
I20260806 22:47:25.114827 17188 leader_election.cc:302] T 77fd972a26eb4f96bdd109505fa77fd1 P a712a3a80999468ea0a596422fb8f3a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a3c36e213304396bf7ebddea9300695, a712a3a80999468ea0a596422fb8f3a1; no voters:
I20260806 22:47:25.114946 17588 raft_consensus.cc:2781] T 77fd972a26eb4f96bdd109505fa77fd1 P a712a3a80999468ea0a596422fb8f3a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:47:25.115126 17588 raft_consensus.cc:686] T 77fd972a26eb4f96bdd109505fa77fd1 P a712a3a80999468ea0a596422fb8f3a1 [term 1 LEADER]: Becoming Leader. State: Replica: a712a3a80999468ea0a596422fb8f3a1, State: Running, Role: LEADER
I20260806 22:47:25.115232 17588 consensus_queue.cc:237] T 77fd972a26eb4f96bdd109505fa77fd1 P a712a3a80999468ea0a596422fb8f3a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a3c36e213304396bf7ebddea9300695" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 38371 } } peers { permanent_uuid: "a712a3a80999468ea0a596422fb8f3a1" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 44463 } } peers { permanent_uuid: "4091dc731c9a4e15b38508539576d38e" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45929 } }
I20260806 22:47:25.115963 16858 catalog_manager.cc:5495] T 77fd972a26eb4f96bdd109505fa77fd1 P a712a3a80999468ea0a596422fb8f3a1 reported cstate change: term changed from 0 to 1, leader changed from <none> to a712a3a80999468ea0a596422fb8f3a1 (127.6.40.66). New cstate: current_term: 1 leader_uuid: "a712a3a80999468ea0a596422fb8f3a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a3c36e213304396bf7ebddea9300695" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 38371 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a712a3a80999468ea0a596422fb8f3a1" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 44463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4091dc731c9a4e15b38508539576d38e" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45929 } health_report { overall_health: UNKNOWN } } }
Aug 06 22:47:25 dist-test-slave-zh2d krb5kdc[16793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056443, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.40.126@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM
I20260806 22:47:25.161131 17629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:25.153617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59696 (local address 127.6.40.126:39333)
0806 22:47:25.153772 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:25.153774 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:25.153787 (+ 13us) server_negotiation.cc:407] Connection header received
0806 22:47:25.153811 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:25.153811 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:25.153821 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:25.153897 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:25.154369 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:25.154875 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:25.155764 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:25.155978 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:25.156045 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:25.158393 (+ 2348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:25.158394 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:25.158394 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:25.158400 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:25.159889 (+ 1489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:25.160260 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:25.160261 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:25.160261 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:25.160318 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:25.160571 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:25.160572 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:25.160572 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:25.160689 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:25.160749 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:25.160949 (+ 200us) server_negotiation.cc:299] Negotiation successful
0806 22:47:25.161056 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":133,"thread_start_us":84,"threads_started":1}
I20260806 22:47:25.161322 17627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:25.153540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:39333 (local address 127.0.0.1:59696)
0806 22:47:25.153660 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:25.153673 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:25.153756 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:25.153914 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:25.153914 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:25.154219 (+ 305us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:25.154267 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:25.154269 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:25.154971 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:25.154972 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:25.155668 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:25.155671 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:25.156055 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:25.156056 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:25.156100 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:25.156694 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:25.156699 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:25.158254 (+ 1555us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:47:25.159969 (+ 1715us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:25.159976 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:25.159978 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:25.160197 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:25.160418 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:25.160419 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:25.160419 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:25.160478 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:25.160757 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:47:25.160764 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:47:25.160807 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 22:47:25.160930 (+ 123us) client_negotiation.cc:240] Negotiation successful
0806 22:47:25.161173 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":86,"mutex_wait_us":116,"thread_start_us":47,"threads_started":1}
I20260806 22:47:25.163483 16858 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.6.40.126@KRBTEST.COM'} at 127.0.0.1:59696:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 22:47:25.163731 16858 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:47:25.165473 17450 tablet_service.cc:1467] Processing CreateTablet for tablet e386d14089eb4f58951141ec10aa0b77 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9347697757b6497ea130626aa465474d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 22:47:25.165498 17257 tablet_service.cc:1467] Processing CreateTablet for tablet e386d14089eb4f58951141ec10aa0b77 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9347697757b6497ea130626aa465474d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 22:47:25.165656 17450 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e386d14089eb4f58951141ec10aa0b77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:47:25.165661 17257 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e386d14089eb4f58951141ec10aa0b77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:47:25.167023 17576 tablet_bootstrap.cc:492] T e386d14089eb4f58951141ec10aa0b77 P 4091dc731c9a4e15b38508539576d38e: Bootstrap starting.
I20260806 22:47:25.167100 17052 tablet_service.cc:1467] Processing CreateTablet for tablet e386d14089eb4f58951141ec10aa0b77 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9347697757b6497ea130626aa465474d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 22:47:25.167260 17052 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e386d14089eb4f58951141ec10aa0b77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:47:25.167472 17576 tablet_bootstrap.cc:654] T e386d14089eb4f58951141ec10aa0b77 P 4091dc731c9a4e15b38508539576d38e: Neither blocks nor log segments found. Creating new log.
I20260806 22:47:25.168148 17576 tablet_bootstrap.cc:492] T e386d14089eb4f58951141ec10aa0b77 P 4091dc731c9a4e15b38508539576d38e: No bootstrap required, opened a new log
I20260806 22:47:25.168212 17576 ts_tablet_manager.cc:1397] T e386d14089eb4f58951141ec10aa0b77 P 4091dc731c9a4e15b38508539576d38e: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260806 22:47:25.168359 17576 raft_consensus.cc:348] T e386d14089eb4f58951141ec10aa0b77 P 4091dc731c9a4e15b38508539576d38e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a3c36e213304396bf7ebddea9300695" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 38371 } } peers { permanent_uuid: "a712a3a80999468ea0a596422fb8f3a1" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 44463 } } peers { permanent_uuid: "4091dc731c9a4e15b38508539576d38e" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45929 } }
I20260806 22:47:25.168435 17576 raft_consensus.cc:374] T e386d14089eb4f58951141ec10aa0b77 P 4091dc731c9a4e15b38508539576d38e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:47:25.168454 17576 raft_consensus.cc:729] T e386d14089eb4f58951141ec10aa0b77 P 4091dc731c9a4e15b38508539576d38e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4091dc731c9a4e15b38508539576d38e, State: Initialized, Role: FOLLOWER
I20260806 22:47:25.168499 17576 consensus_queue.cc:260] T e386d14089eb4f58951141ec10aa0b77 P 4091dc731c9a4e15b38508539576d38e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a3c36e213304396bf7ebddea9300695" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 38371 } } peers { permanent_uuid: "a712a3a80999468ea0a596422fb8f3a1" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 44463 } } peers { permanent_uuid: "4091dc731c9a4e15b38508539576d38e" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45929 } }
I20260806 22:47:25.168612 17576 ts_tablet_manager.cc:1428] T e386d14089eb4f58951141ec10aa0b77 P 4091dc731c9a4e15b38508539576d38e: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260806 22:47:25.169003 17577 tablet_bootstrap.cc:492] T e386d14089eb4f58951141ec10aa0b77 P 1a3c36e213304396bf7ebddea9300695: Bootstrap starting.
I20260806 22:47:25.169036 17581 tablet_bootstrap.cc:492] T e386d14089eb4f58951141ec10aa0b77 P a712a3a80999468ea0a596422fb8f3a1: Bootstrap starting.
I20260806 22:47:25.169312 17577 tablet_bootstrap.cc:654] T e386d14089eb4f58951141ec10aa0b77 P 1a3c36e213304396bf7ebddea9300695: Neither blocks nor log segments found. Creating new log.
I20260806 22:47:25.169426 17581 tablet_bootstrap.cc:654] T e386d14089eb4f58951141ec10aa0b77 P a712a3a80999468ea0a596422fb8f3a1: Neither blocks nor log segments found. Creating new log.
I20260806 22:47:25.169866 17586 tablet_replica.cc:439] T e386d14089eb4f58951141ec10aa0b77 P 4091dc731c9a4e15b38508539576d38e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a3c36e213304396bf7ebddea9300695" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 38371 } } peers { permanent_uuid: "a712a3a80999468ea0a596422fb8f3a1" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 44463 } } peers { permanent_uuid: "4091dc731c9a4e15b38508539576d38e" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45929 } } }
I20260806 22:47:25.169965 17577 tablet_bootstrap.cc:492] T e386d14089eb4f58951141ec10aa0b77 P 1a3c36e213304396bf7ebddea9300695: No bootstrap required, opened a new log
I20260806 22:47:25.169974 17586 tablet_replica.cc:442] T e386d14089eb4f58951141ec10aa0b77 P 4091dc731c9a4e15b38508539576d38e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:47:25.170027 17577 ts_tablet_manager.cc:1397] T e386d14089eb4f58951141ec10aa0b77 P 1a3c36e213304396bf7ebddea9300695: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 22:47:25.170181 17577 raft_consensus.cc:348] T e386d14089eb4f58951141ec10aa0b77 P 1a3c36e213304396bf7ebddea9300695 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a3c36e213304396bf7ebddea9300695" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 38371 } } peers { permanent_uuid: "a712a3a80999468ea0a596422fb8f3a1" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 44463 } } peers { permanent_uuid: "4091dc731c9a4e15b38508539576d38e" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45929 } }
I20260806 22:47:25.170261 17577 raft_consensus.cc:374] T e386d14089eb4f58951141ec10aa0b77 P 1a3c36e213304396bf7ebddea9300695 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:47:25.170284 17577 raft_consensus.cc:729] T e386d14089eb4f58951141ec10aa0b77 P 1a3c36e213304396bf7ebddea9300695 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a3c36e213304396bf7ebddea9300695, State: Initialized, Role: FOLLOWER
I20260806 22:47:25.170388 17581 tablet_bootstrap.cc:492] T e386d14089eb4f58951141ec10aa0b77 P a712a3a80999468ea0a596422fb8f3a1: No bootstrap required, opened a new log
I20260806 22:47:25.170396 17577 consensus_queue.cc:260] T e386d14089eb4f58951141ec10aa0b77 P 1a3c36e213304396bf7ebddea9300695 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a3c36e213304396bf7ebddea9300695" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 38371 } } peers { permanent_uuid: "a712a3a80999468ea0a596422fb8f3a1" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 44463 } } peers { permanent_uuid: "4091dc731c9a4e15b38508539576d38e" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45929 } }
I20260806 22:47:25.170470 17581 ts_tablet_manager.cc:1397] T e386d14089eb4f58951141ec10aa0b77 P a712a3a80999468ea0a596422fb8f3a1: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 22:47:25.170532 17577 ts_tablet_manager.cc:1428] T e386d14089eb4f58951141ec10aa0b77 P 1a3c36e213304396bf7ebddea9300695: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 22:47:25.170634 17581 raft_consensus.cc:348] T e386d14089eb4f58951141ec10aa0b77 P a712a3a80999468ea0a596422fb8f3a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a3c36e213304396bf7ebddea9300695" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 38371 } } peers { permanent_uuid: "a712a3a80999468ea0a596422fb8f3a1" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 44463 } } peers { permanent_uuid: "4091dc731c9a4e15b38508539576d38e" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45929 } }
I20260806 22:47:25.170647 17594 tablet_replica.cc:439] T e386d14089eb4f58951141ec10aa0b77 P 1a3c36e213304396bf7ebddea9300695: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a3c36e213304396bf7ebddea9300695" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 38371 } } peers { permanent_uuid: "a712a3a80999468ea0a596422fb8f3a1" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 44463 } } peers { permanent_uuid: "4091dc731c9a4e15b38508539576d38e" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45929 } } }
I20260806 22:47:25.170709 17594 tablet_replica.cc:442] T e386d14089eb4f58951141ec10aa0b77 P 1a3c36e213304396bf7ebddea9300695: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:47:25.170727 17581 raft_consensus.cc:374] T e386d14089eb4f58951141ec10aa0b77 P a712a3a80999468ea0a596422fb8f3a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:47:25.170753 17581 raft_consensus.cc:729] T e386d14089eb4f58951141ec10aa0b77 P a712a3a80999468ea0a596422fb8f3a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a712a3a80999468ea0a596422fb8f3a1, State: Initialized, Role: FOLLOWER
I20260806 22:47:25.170806 17581 consensus_queue.cc:260] T e386d14089eb4f58951141ec10aa0b77 P a712a3a80999468ea0a596422fb8f3a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a3c36e213304396bf7ebddea9300695" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 38371 } } peers { permanent_uuid: "a712a3a80999468ea0a596422fb8f3a1" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 44463 } } peers { permanent_uuid: "4091dc731c9a4e15b38508539576d38e" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45929 } }
I20260806 22:47:25.170905 17581 ts_tablet_manager.cc:1428] T e386d14089eb4f58951141ec10aa0b77 P a712a3a80999468ea0a596422fb8f3a1: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:25.170962 17588 tablet_replica.cc:439] T e386d14089eb4f58951141ec10aa0b77 P a712a3a80999468ea0a596422fb8f3a1: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a3c36e213304396bf7ebddea9300695" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 38371 } } peers { permanent_uuid: "a712a3a80999468ea0a596422fb8f3a1" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 44463 } } peers { permanent_uuid: "4091dc731c9a4e15b38508539576d38e" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45929 } } }
I20260806 22:47:25.171074 17588 tablet_replica.cc:442] T e386d14089eb4f58951141ec10aa0b77 P a712a3a80999468ea0a596422fb8f3a1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:47:25.241065 17586 raft_consensus.cc:482] T e386d14089eb4f58951141ec10aa0b77 P 4091dc731c9a4e15b38508539576d38e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:47:25.241178 17586 raft_consensus.cc:504] T e386d14089eb4f58951141ec10aa0b77 P 4091dc731c9a4e15b38508539576d38e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a3c36e213304396bf7ebddea9300695" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 38371 } } peers { permanent_uuid: "a712a3a80999468ea0a596422fb8f3a1" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 44463 } } peers { permanent_uuid: "4091dc731c9a4e15b38508539576d38e" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45929 } }
I20260806 22:47:25.241410 17586 leader_election.cc:288] T e386d14089eb4f58951141ec10aa0b77 P 4091dc731c9a4e15b38508539576d38e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1a3c36e213304396bf7ebddea9300695 (127.6.40.65:38371), a712a3a80999468ea0a596422fb8f3a1 (127.6.40.66:44463)
I20260806 22:47:25.241580 17057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e386d14089eb4f58951141ec10aa0b77" candidate_uuid: "4091dc731c9a4e15b38508539576d38e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a3c36e213304396bf7ebddea9300695" is_pre_election: true
I20260806 22:47:25.241717 17057 raft_consensus.cc:2454] T e386d14089eb4f58951141ec10aa0b77 P 1a3c36e213304396bf7ebddea9300695 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4091dc731c9a4e15b38508539576d38e in term 0.
I20260806 22:47:25.241751 17286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e386d14089eb4f58951141ec10aa0b77" candidate_uuid: "4091dc731c9a4e15b38508539576d38e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a712a3a80999468ea0a596422fb8f3a1" is_pre_election: true
I20260806 22:47:25.241887 17286 raft_consensus.cc:2454] T e386d14089eb4f58951141ec10aa0b77 P a712a3a80999468ea0a596422fb8f3a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4091dc731c9a4e15b38508539576d38e in term 0.
I20260806 22:47:25.241914 17382 leader_election.cc:302] T e386d14089eb4f58951141ec10aa0b77 P 4091dc731c9a4e15b38508539576d38e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a3c36e213304396bf7ebddea9300695, 4091dc731c9a4e15b38508539576d38e; no voters:
I20260806 22:47:25.242024 17586 raft_consensus.cc:2781] T e386d14089eb4f58951141ec10aa0b77 P 4091dc731c9a4e15b38508539576d38e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:47:25.242081 17586 raft_consensus.cc:482] T e386d14089eb4f58951141ec10aa0b77 P 4091dc731c9a4e15b38508539576d38e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:47:25.242112 17586 raft_consensus.cc:3037] T e386d14089eb4f58951141ec10aa0b77 P 4091dc731c9a4e15b38508539576d38e [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:47:25.242990 17586 raft_consensus.cc:504] T e386d14089eb4f58951141ec10aa0b77 P 4091dc731c9a4e15b38508539576d38e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a3c36e213304396bf7ebddea9300695" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 38371 } } peers { permanent_uuid: "a712a3a80999468ea0a596422fb8f3a1" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 44463 } } peers { permanent_uuid: "4091dc731c9a4e15b38508539576d38e" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45929 } }
I20260806 22:47:25.243151 17586 leader_election.cc:288] T e386d14089eb4f58951141ec10aa0b77 P 4091dc731c9a4e15b38508539576d38e [CANDIDATE]: Term 1 election: Requested vote from peers 1a3c36e213304396bf7ebddea9300695 (127.6.40.65:38371), a712a3a80999468ea0a596422fb8f3a1 (127.6.40.66:44463)
I20260806 22:47:25.243340 17057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e386d14089eb4f58951141ec10aa0b77" candidate_uuid: "4091dc731c9a4e15b38508539576d38e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a3c36e213304396bf7ebddea9300695"
I20260806 22:47:25.243369 17286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e386d14089eb4f58951141ec10aa0b77" candidate_uuid: "4091dc731c9a4e15b38508539576d38e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a712a3a80999468ea0a596422fb8f3a1"
I20260806 22:47:25.243489 17286 raft_consensus.cc:3037] T e386d14089eb4f58951141ec10aa0b77 P a712a3a80999468ea0a596422fb8f3a1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:47:25.243527 17057 raft_consensus.cc:3037] T e386d14089eb4f58951141ec10aa0b77 P 1a3c36e213304396bf7ebddea9300695 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:47:25.244197 17286 raft_consensus.cc:2454] T e386d14089eb4f58951141ec10aa0b77 P a712a3a80999468ea0a596422fb8f3a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4091dc731c9a4e15b38508539576d38e in term 1.
I20260806 22:47:25.244390 17385 leader_election.cc:302] T e386d14089eb4f58951141ec10aa0b77 P 4091dc731c9a4e15b38508539576d38e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4091dc731c9a4e15b38508539576d38e, a712a3a80999468ea0a596422fb8f3a1; no voters:
I20260806 22:47:25.244494 17586 raft_consensus.cc:2781] T e386d14089eb4f58951141ec10aa0b77 P 4091dc731c9a4e15b38508539576d38e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:47:25.244544 17057 raft_consensus.cc:2454] T e386d14089eb4f58951141ec10aa0b77 P 1a3c36e213304396bf7ebddea9300695 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4091dc731c9a4e15b38508539576d38e in term 1.
I20260806 22:47:25.244623 17586 raft_consensus.cc:686] T e386d14089eb4f58951141ec10aa0b77 P 4091dc731c9a4e15b38508539576d38e [term 1 LEADER]: Becoming Leader. State: Replica: 4091dc731c9a4e15b38508539576d38e, State: Running, Role: LEADER
I20260806 22:47:25.244700 17586 consensus_queue.cc:237] T e386d14089eb4f58951141ec10aa0b77 P 4091dc731c9a4e15b38508539576d38e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a3c36e213304396bf7ebddea9300695" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 38371 } } peers { permanent_uuid: "a712a3a80999468ea0a596422fb8f3a1" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 44463 } } peers { permanent_uuid: "4091dc731c9a4e15b38508539576d38e" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45929 } }
I20260806 22:47:25.245133 17634 tablet_replica.cc:439] T e386d14089eb4f58951141ec10aa0b77 P 4091dc731c9a4e15b38508539576d38e: TxnStatusTablet state changed. Reason: New leader 4091dc731c9a4e15b38508539576d38e. Latest consensus state: current_term: 1 leader_uuid: "4091dc731c9a4e15b38508539576d38e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a3c36e213304396bf7ebddea9300695" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 38371 } } peers { permanent_uuid: "a712a3a80999468ea0a596422fb8f3a1" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 44463 } } peers { permanent_uuid: "4091dc731c9a4e15b38508539576d38e" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45929 } } }
I20260806 22:47:25.245285 17634 tablet_replica.cc:442] T e386d14089eb4f58951141ec10aa0b77 P 4091dc731c9a4e15b38508539576d38e: This TxnStatusTablet replica's current role is: LEADER
I20260806 22:47:25.245505 17638 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 22:47:25.245651 17638 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 22:47:25.245669 16857 catalog_manager.cc:5495] T e386d14089eb4f58951141ec10aa0b77 P 4091dc731c9a4e15b38508539576d38e reported cstate change: term changed from 0 to 1, leader changed from <none> to 4091dc731c9a4e15b38508539576d38e (127.6.40.67). New cstate: current_term: 1 leader_uuid: "4091dc731c9a4e15b38508539576d38e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a3c36e213304396bf7ebddea9300695" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 38371 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a712a3a80999468ea0a596422fb8f3a1" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 44463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4091dc731c9a4e15b38508539576d38e" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45929 } health_report { overall_health: HEALTHY } } }
I20260806 22:47:25.264328 17574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:25.261751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37124 (local address 127.6.40.67:45929)
0806 22:47:25.261822 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:25.261825 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:25.261835 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:47:25.261952 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:25.261953 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:25.261981 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:25.262067 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:47:25.262208 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:25.262717 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:25.263496 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:25.263689 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:25.263775 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:25.263988 (+ 213us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:47:25.264089 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:47:25.264172 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:25.264231 (+ 59us) server_negotiation.cc:299] Negotiation successful
0806 22:47:25.264252 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 22:47:25.264354 17627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:25.261673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.67:45929 (local address 127.0.0.1:37124)
0806 22:47:25.261750 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:25.261765 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:25.261862 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:25.262077 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:25.262078 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:25.262080 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 22:47:25.262134 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:25.262137 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:25.262809 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:25.262810 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:25.263406 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:25.263408 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:25.263776 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:25.263778 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:25.263859 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:25.263874 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 22:47:25.264158 (+ 284us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 22:47:25.264169 (+ 11us) client_negotiation.cc:769] Sending connection context
0806 22:47:25.264225 (+ 56us) client_negotiation.cc:240] Negotiation successful
0806 22:47:25.264284 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260806 22:47:25.265796 17286 raft_consensus.cc:1261] T e386d14089eb4f58951141ec10aa0b77 P a712a3a80999468ea0a596422fb8f3a1 [term 1 FOLLOWER]: Refusing update from remote peer 4091dc731c9a4e15b38508539576d38e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:47:25.265955 17057 raft_consensus.cc:1261] T e386d14089eb4f58951141ec10aa0b77 P 1a3c36e213304396bf7ebddea9300695 [term 1 FOLLOWER]: Refusing update from remote peer 4091dc731c9a4e15b38508539576d38e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:47:25.266085 17634 consensus_queue.cc:1035] T e386d14089eb4f58951141ec10aa0b77 P 4091dc731c9a4e15b38508539576d38e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a712a3a80999468ea0a596422fb8f3a1" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:47:25.266259 17586 consensus_queue.cc:1035] T e386d14089eb4f58951141ec10aa0b77 P 4091dc731c9a4e15b38508539576d38e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a3c36e213304396bf7ebddea9300695" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 38371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:47:25.266929 17588 tablet_replica.cc:439] T e386d14089eb4f58951141ec10aa0b77 P a712a3a80999468ea0a596422fb8f3a1: TxnStatusTablet state changed. Reason: New leader 4091dc731c9a4e15b38508539576d38e. Latest consensus state: current_term: 1 leader_uuid: "4091dc731c9a4e15b38508539576d38e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a3c36e213304396bf7ebddea9300695" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 38371 } } peers { permanent_uuid: "a712a3a80999468ea0a596422fb8f3a1" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 44463 } } peers { permanent_uuid: "4091dc731c9a4e15b38508539576d38e" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45929 } } }
I20260806 22:47:25.267028 17588 tablet_replica.cc:442] T e386d14089eb4f58951141ec10aa0b77 P a712a3a80999468ea0a596422fb8f3a1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:47:25.267123 17594 tablet_replica.cc:439] T e386d14089eb4f58951141ec10aa0b77 P 1a3c36e213304396bf7ebddea9300695: TxnStatusTablet state changed. Reason: New leader 4091dc731c9a4e15b38508539576d38e. Latest consensus state: current_term: 1 leader_uuid: "4091dc731c9a4e15b38508539576d38e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a3c36e213304396bf7ebddea9300695" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 38371 } } peers { permanent_uuid: "a712a3a80999468ea0a596422fb8f3a1" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 44463 } } peers { permanent_uuid: "4091dc731c9a4e15b38508539576d38e" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45929 } } }
I20260806 22:47:25.267230 17594 tablet_replica.cc:442] T e386d14089eb4f58951141ec10aa0b77 P 1a3c36e213304396bf7ebddea9300695: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:47:25.267915 17634 tablet_replica.cc:439] T e386d14089eb4f58951141ec10aa0b77 P 4091dc731c9a4e15b38508539576d38e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4091dc731c9a4e15b38508539576d38e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a3c36e213304396bf7ebddea9300695" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 38371 } } peers { permanent_uuid: "a712a3a80999468ea0a596422fb8f3a1" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 44463 } } peers { permanent_uuid: "4091dc731c9a4e15b38508539576d38e" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45929 } } }
I20260806 22:47:25.268006 17634 tablet_replica.cc:442] T e386d14089eb4f58951141ec10aa0b77 P 4091dc731c9a4e15b38508539576d38e: This TxnStatusTablet replica's current role is: LEADER
I20260806 22:47:25.268276 17594 tablet_replica.cc:439] T e386d14089eb4f58951141ec10aa0b77 P 1a3c36e213304396bf7ebddea9300695: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4091dc731c9a4e15b38508539576d38e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a3c36e213304396bf7ebddea9300695" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 38371 } } peers { permanent_uuid: "a712a3a80999468ea0a596422fb8f3a1" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 44463 } } peers { permanent_uuid: "4091dc731c9a4e15b38508539576d38e" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45929 } } }
I20260806 22:47:25.268376 17594 tablet_replica.cc:442] T e386d14089eb4f58951141ec10aa0b77 P 1a3c36e213304396bf7ebddea9300695: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:47:25.269033 17586 tablet_replica.cc:439] T e386d14089eb4f58951141ec10aa0b77 P 4091dc731c9a4e15b38508539576d38e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4091dc731c9a4e15b38508539576d38e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a3c36e213304396bf7ebddea9300695" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 38371 } } peers { permanent_uuid: "a712a3a80999468ea0a596422fb8f3a1" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 44463 } } peers { permanent_uuid: "4091dc731c9a4e15b38508539576d38e" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45929 } } }
I20260806 22:47:25.269130 17586 tablet_replica.cc:442] T e386d14089eb4f58951141ec10aa0b77 P 4091dc731c9a4e15b38508539576d38e: This TxnStatusTablet replica's current role is: LEADER
I20260806 22:47:25.269619 17600 mvcc.cc:205] Tried to move back new op lower bound from 7315687199806582784 to 7315687199723184128. Current Snapshot: MvccSnapshot[applied={T|T < 7315687199806582784 or (T in {7315687199806582784})}]
I20260806 22:47:25.269862 17588 tablet_replica.cc:439] T e386d14089eb4f58951141ec10aa0b77 P a712a3a80999468ea0a596422fb8f3a1: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4091dc731c9a4e15b38508539576d38e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a3c36e213304396bf7ebddea9300695" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 38371 } } peers { permanent_uuid: "a712a3a80999468ea0a596422fb8f3a1" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 44463 } } peers { permanent_uuid: "4091dc731c9a4e15b38508539576d38e" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45929 } } }
I20260806 22:47:25.269963 17588 tablet_replica.cc:442] T e386d14089eb4f58951141ec10aa0b77 P a712a3a80999468ea0a596422fb8f3a1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:47:25.276070 17573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:25.269855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57434 (local address 127.6.40.66:44463)
0806 22:47:25.269910 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:25.269912 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:25.269970 (+ 58us) server_negotiation.cc:407] Connection header received
0806 22:47:25.270062 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:25.270063 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:25.270086 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:25.270172 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:47:25.270390 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:25.270856 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:25.272153 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:25.272486 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:25.272560 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:25.272787 (+ 227us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:47:25.272994 (+ 207us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:47:25.273097 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:25.275920 (+ 2823us) server_negotiation.cc:299] Negotiation successful
0806 22:47:25.275951 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 22:47:25.288734 17574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:25.285920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.66:57499 (local address 127.6.40.67:45929)
0806 22:47:25.285966 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:25.285968 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:25.285976 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:47:25.286003 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:25.286004 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:25.286016 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:25.286064 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:47:25.286305 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:25.286952 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:25.288015 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:25.288223 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:25.288298 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:25.288480 (+ 182us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:47:25.288515 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:47:25.288571 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:25.288629 (+ 58us) server_negotiation.cc:299] Negotiation successful
0806 22:47:25.288653 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 22:47:25.288774 17648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:25.285709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.67:45929 (local address 127.6.40.66:57499)
0806 22:47:25.285864 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:25.285877 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:25.285936 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:25.286113 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:25.286114 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:25.286116 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 22:47:25.286209 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:25.286211 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:25.287036 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:25.287038 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:25.287921 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:25.287924 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:25.288311 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:25.288312 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:25.288384 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:25.288398 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 22:47:25.288581 (+ 183us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 22:47:25.288583 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 22:47:25.288627 (+ 44us) client_negotiation.cc:240] Negotiation successful
0806 22:47:25.288673 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":83,"threads_started":1}
I20260806 22:47:25.291415 17057 raft_consensus.cc:1261] T 77fd972a26eb4f96bdd109505fa77fd1 P 1a3c36e213304396bf7ebddea9300695 [term 1 FOLLOWER]: Refusing update from remote peer a712a3a80999468ea0a596422fb8f3a1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:47:25.291603 17476 raft_consensus.cc:1261] T 77fd972a26eb4f96bdd109505fa77fd1 P 4091dc731c9a4e15b38508539576d38e [term 1 FOLLOWER]: Refusing update from remote peer a712a3a80999468ea0a596422fb8f3a1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:47:25.291713 17618 consensus_queue.cc:1035] T 77fd972a26eb4f96bdd109505fa77fd1 P a712a3a80999468ea0a596422fb8f3a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a3c36e213304396bf7ebddea9300695" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 38371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:47:25.291922 17618 consensus_queue.cc:1035] T 77fd972a26eb4f96bdd109505fa77fd1 P a712a3a80999468ea0a596422fb8f3a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4091dc731c9a4e15b38508539576d38e" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:47:25.301083 17653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:25.297964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.66:44463 (local address 127.6.40.67:44673)
0806 22:47:25.298168 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:25.298178 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:25.298236 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:25.298359 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:25.298360 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:25.298361 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 22:47:25.298416 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:25.298417 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:25.299229 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:25.299230 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:25.300083 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:25.300086 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:25.300599 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:25.300600 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:25.300645 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:25.300658 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 22:47:25.300910 (+ 252us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 22:47:25.300912 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 22:47:25.300966 (+ 54us) client_negotiation.cc:240] Negotiation successful
0806 22:47:25.301002 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":77,"threads_started":1}
I20260806 22:47:25.301072 17573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:25.298022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.67:44673 (local address 127.6.40.66:44463)
0806 22:47:25.298066 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:25.298077 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:25.298244 (+ 167us) server_negotiation.cc:407] Connection header received
0806 22:47:25.298273 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:25.298274 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:25.298285 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:25.298359 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:47:25.298454 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:25.299143 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:25.300170 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:25.300504 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:25.300578 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:25.300760 (+ 182us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:47:25.300794 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:47:25.300883 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:25.300975 (+ 92us) server_negotiation.cc:299] Negotiation successful
0806 22:47:25.300997 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 22:47:25.310281 16858 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59688:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:47:25.310398 16858 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59688:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:47:25.311084 16858 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a023cd9df99d4e3db64a1b4977884980: Sending DeleteTablet for 3 replicas of tablet 77fd972a26eb4f96bdd109505fa77fd1
I20260806 22:47:25.311378 17257 tablet_service.cc:1514] Processing DeleteTablet for tablet 77fd972a26eb4f96bdd109505fa77fd1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:47:25 UTC) from {username='oryx', principal='oryx/127.6.40.126@KRBTEST.COM'} at 127.0.0.1:57432
I20260806 22:47:25.311473 17609 tablet_replica.cc:330] T 77fd972a26eb4f96bdd109505fa77fd1 P a712a3a80999468ea0a596422fb8f3a1: stopping tablet replica
I20260806 22:47:25.311522 17609 raft_consensus.cc:2229] T 77fd972a26eb4f96bdd109505fa77fd1 P a712a3a80999468ea0a596422fb8f3a1 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:47:25.311568 17450 tablet_service.cc:1514] Processing DeleteTablet for tablet 77fd972a26eb4f96bdd109505fa77fd1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:47:25 UTC) from {username='oryx', principal='oryx/127.6.40.126@KRBTEST.COM'} at 127.0.0.1:37116
I20260806 22:47:25.311609 17609 raft_consensus.cc:2258] T 77fd972a26eb4f96bdd109505fa77fd1 P a712a3a80999468ea0a596422fb8f3a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:47:25.311666 17610 tablet_replica.cc:330] T 77fd972a26eb4f96bdd109505fa77fd1 P 4091dc731c9a4e15b38508539576d38e: stopping tablet replica
I20260806 22:47:25.311712 17610 raft_consensus.cc:2229] T 77fd972a26eb4f96bdd109505fa77fd1 P 4091dc731c9a4e15b38508539576d38e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:47:25.311754 17610 raft_consensus.cc:2258] T 77fd972a26eb4f96bdd109505fa77fd1 P 4091dc731c9a4e15b38508539576d38e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:47:25.312202 17610 ts_tablet_manager.cc:1905] T 77fd972a26eb4f96bdd109505fa77fd1 P 4091dc731c9a4e15b38508539576d38e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:47:25.313064 17609 ts_tablet_manager.cc:1905] T 77fd972a26eb4f96bdd109505fa77fd1 P a712a3a80999468ea0a596422fb8f3a1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:47:25.314478 17610 ts_tablet_manager.cc:1918] T 77fd972a26eb4f96bdd109505fa77fd1 P 4091dc731c9a4e15b38508539576d38e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:47:25.314540 17610 log.cc:1196] T 77fd972a26eb4f96bdd109505fa77fd1 P 4091dc731c9a4e15b38508539576d38e: Deleting WAL directory at /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/minicluster-data/ts-2/wal/wals/77fd972a26eb4f96bdd109505fa77fd1
I20260806 22:47:25.314769 17610 ts_tablet_manager.cc:1939] T 77fd972a26eb4f96bdd109505fa77fd1 P 4091dc731c9a4e15b38508539576d38e: Deleting consensus metadata
I20260806 22:47:25.314889 17609 ts_tablet_manager.cc:1918] T 77fd972a26eb4f96bdd109505fa77fd1 P a712a3a80999468ea0a596422fb8f3a1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:47:25.314944 17609 log.cc:1196] T 77fd972a26eb4f96bdd109505fa77fd1 P a712a3a80999468ea0a596422fb8f3a1: Deleting WAL directory at /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/minicluster-data/ts-1/wal/wals/77fd972a26eb4f96bdd109505fa77fd1
I20260806 22:47:25.315101 17052 tablet_service.cc:1514] Processing DeleteTablet for tablet 77fd972a26eb4f96bdd109505fa77fd1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:47:25 UTC) from {username='oryx', principal='oryx/127.6.40.126@KRBTEST.COM'} at 127.0.0.1:35748
I20260806 22:47:25.315192 17609 ts_tablet_manager.cc:1939] T 77fd972a26eb4f96bdd109505fa77fd1 P a712a3a80999468ea0a596422fb8f3a1: Deleting consensus metadata
I20260806 22:47:25.315201 17611 tablet_replica.cc:330] T 77fd972a26eb4f96bdd109505fa77fd1 P 1a3c36e213304396bf7ebddea9300695: stopping tablet replica
I20260806 22:47:25.315264 17611 raft_consensus.cc:2229] T 77fd972a26eb4f96bdd109505fa77fd1 P 1a3c36e213304396bf7ebddea9300695 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:47:25.315310 17611 raft_consensus.cc:2258] T 77fd972a26eb4f96bdd109505fa77fd1 P 1a3c36e213304396bf7ebddea9300695 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:47:25.315294 16838 catalog_manager.cc:4841] TS 4091dc731c9a4e15b38508539576d38e (127.6.40.67:45929): tablet 77fd972a26eb4f96bdd109505fa77fd1 (table test-table [id=3dc395d5c14348f7a4c4a067a54f4e58]) successfully deleted
I20260806 22:47:25.315672 17611 ts_tablet_manager.cc:1905] T 77fd972a26eb4f96bdd109505fa77fd1 P 1a3c36e213304396bf7ebddea9300695: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:47:25.317790 17611 ts_tablet_manager.cc:1918] T 77fd972a26eb4f96bdd109505fa77fd1 P 1a3c36e213304396bf7ebddea9300695: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:47:25.317869 17611 log.cc:1196] T 77fd972a26eb4f96bdd109505fa77fd1 P 1a3c36e213304396bf7ebddea9300695: Deleting WAL directory at /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/minicluster-data/ts-0/wal/wals/77fd972a26eb4f96bdd109505fa77fd1
I20260806 22:47:25.318105 17611 ts_tablet_manager.cc:1939] T 77fd972a26eb4f96bdd109505fa77fd1 P 1a3c36e213304396bf7ebddea9300695: Deleting consensus metadata
I20260806 22:47:25.318805 16834 catalog_manager.cc:4841] TS 1a3c36e213304396bf7ebddea9300695 (127.6.40.65:38371): tablet 77fd972a26eb4f96bdd109505fa77fd1 (table test-table [id=3dc395d5c14348f7a4c4a067a54f4e58]) successfully deleted
I20260806 22:47:25.318914 16838 catalog_manager.cc:4841] TS a712a3a80999468ea0a596422fb8f3a1 (127.6.40.66:44463): tablet 77fd972a26eb4f96bdd109505fa77fd1 (table test-table [id=3dc395d5c14348f7a4c4a067a54f4e58]) successfully deleted
Aug 06 22:47:25 dist-test-slave-zh2d krb5kdc[16793](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056445, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 22:47:25 dist-test-slave-zh2d krb5kdc[16793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:47:25 dist-test-slave-zh2d krb5kdc[16793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
W20260806 22:47:25.348778 17629 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:47:25.341413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59700 (local address 127.6.40.126:39333)
0806 22:47:25.341492 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:25.341495 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:25.341549 (+ 54us) server_negotiation.cc:407] Connection header received
0806 22:47:25.341639 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:25.341640 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:25.341651 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:25.341717 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:25.342530 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:25.343342 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:25.344536 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:25.344915 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:25.344960 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:25.348695 (+ 3735us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59700: BlockingRecv error: recv got EOF from 127.0.0.1:59700 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 06 22:47:25 dist-test-slave-zh2d krb5kdc[16793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056445, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM
I20260806 22:47:25.359514 17629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:25.350357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59704 (local address 127.6.40.126:39333)
0806 22:47:25.350404 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:25.350406 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:25.350447 (+ 41us) server_negotiation.cc:407] Connection header received
0806 22:47:25.350475 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:25.350476 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:25.350487 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:25.350545 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:25.351168 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:25.351892 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:25.352994 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:25.353300 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:25.353393 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:25.356254 (+ 2861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:25.356256 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:25.356256 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:25.356264 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:25.358088 (+ 1824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:25.358578 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:25.358579 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:25.358580 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:25.358648 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:25.358927 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:25.358929 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:25.358929 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:25.359076 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:25.359194 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:25.359333 (+ 139us) server_negotiation.cc:299] Negotiation successful
0806 22:47:25.359419 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 22:47:25.361266 16858 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59704:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:47:25.361469 16858 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:47:25.362943 17052 tablet_service.cc:1467] Processing CreateTablet for tablet 1949bad265884b36b5dec4d0842d0964 (DEFAULT_TABLE table=test-table [id=64d259b6d28242ba829f5957beb81244]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:47:25.363107 17052 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1949bad265884b36b5dec4d0842d0964. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:47:25.363119 17257 tablet_service.cc:1467] Processing CreateTablet for tablet 1949bad265884b36b5dec4d0842d0964 (DEFAULT_TABLE table=test-table [id=64d259b6d28242ba829f5957beb81244]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:47:25.363291 17257 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1949bad265884b36b5dec4d0842d0964. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:47:25.364689 17450 tablet_service.cc:1467] Processing CreateTablet for tablet 1949bad265884b36b5dec4d0842d0964 (DEFAULT_TABLE table=test-table [id=64d259b6d28242ba829f5957beb81244]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:47:25.364722 17577 tablet_bootstrap.cc:492] T 1949bad265884b36b5dec4d0842d0964 P 1a3c36e213304396bf7ebddea9300695: Bootstrap starting.
I20260806 22:47:25.364817 17450 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1949bad265884b36b5dec4d0842d0964. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:47:25.365290 17577 tablet_bootstrap.cc:654] T 1949bad265884b36b5dec4d0842d0964 P 1a3c36e213304396bf7ebddea9300695: Neither blocks nor log segments found. Creating new log.
I20260806 22:47:25.366365 17581 tablet_bootstrap.cc:492] T 1949bad265884b36b5dec4d0842d0964 P a712a3a80999468ea0a596422fb8f3a1: Bootstrap starting.
I20260806 22:47:25.366797 17581 tablet_bootstrap.cc:654] T 1949bad265884b36b5dec4d0842d0964 P a712a3a80999468ea0a596422fb8f3a1: Neither blocks nor log segments found. Creating new log.
I20260806 22:47:25.367169 17576 tablet_bootstrap.cc:492] T 1949bad265884b36b5dec4d0842d0964 P 4091dc731c9a4e15b38508539576d38e: Bootstrap starting.
I20260806 22:47:25.367604 17576 tablet_bootstrap.cc:654] T 1949bad265884b36b5dec4d0842d0964 P 4091dc731c9a4e15b38508539576d38e: Neither blocks nor log segments found. Creating new log.
I20260806 22:47:25.369026 17581 tablet_bootstrap.cc:492] T 1949bad265884b36b5dec4d0842d0964 P a712a3a80999468ea0a596422fb8f3a1: No bootstrap required, opened a new log
I20260806 22:47:25.369084 17581 ts_tablet_manager.cc:1397] T 1949bad265884b36b5dec4d0842d0964 P a712a3a80999468ea0a596422fb8f3a1: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260806 22:47:25.369086 17576 tablet_bootstrap.cc:492] T 1949bad265884b36b5dec4d0842d0964 P 4091dc731c9a4e15b38508539576d38e: No bootstrap required, opened a new log
I20260806 22:47:25.369133 17576 ts_tablet_manager.cc:1397] T 1949bad265884b36b5dec4d0842d0964 P 4091dc731c9a4e15b38508539576d38e: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 22:47:25.369230 17581 raft_consensus.cc:348] T 1949bad265884b36b5dec4d0842d0964 P a712a3a80999468ea0a596422fb8f3a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a3c36e213304396bf7ebddea9300695" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 38371 } } peers { permanent_uuid: "a712a3a80999468ea0a596422fb8f3a1" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 44463 } } peers { permanent_uuid: "4091dc731c9a4e15b38508539576d38e" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45929 } }
I20260806 22:47:25.369302 17581 raft_consensus.cc:374] T 1949bad265884b36b5dec4d0842d0964 P a712a3a80999468ea0a596422fb8f3a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:47:25.369321 17581 raft_consensus.cc:729] T 1949bad265884b36b5dec4d0842d0964 P a712a3a80999468ea0a596422fb8f3a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a712a3a80999468ea0a596422fb8f3a1, State: Initialized, Role: FOLLOWER
I20260806 22:47:25.369328 17576 raft_consensus.cc:348] T 1949bad265884b36b5dec4d0842d0964 P 4091dc731c9a4e15b38508539576d38e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a3c36e213304396bf7ebddea9300695" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 38371 } } peers { permanent_uuid: "a712a3a80999468ea0a596422fb8f3a1" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 44463 } } peers { permanent_uuid: "4091dc731c9a4e15b38508539576d38e" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45929 } }
I20260806 22:47:25.369375 17581 consensus_queue.cc:260] T 1949bad265884b36b5dec4d0842d0964 P a712a3a80999468ea0a596422fb8f3a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a3c36e213304396bf7ebddea9300695" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 38371 } } peers { permanent_uuid: "a712a3a80999468ea0a596422fb8f3a1" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 44463 } } peers { permanent_uuid: "4091dc731c9a4e15b38508539576d38e" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45929 } }
I20260806 22:47:25.369400 17576 raft_consensus.cc:374] T 1949bad265884b36b5dec4d0842d0964 P 4091dc731c9a4e15b38508539576d38e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:47:25.369422 17576 raft_consensus.cc:729] T 1949bad265884b36b5dec4d0842d0964 P 4091dc731c9a4e15b38508539576d38e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4091dc731c9a4e15b38508539576d38e, State: Initialized, Role: FOLLOWER
I20260806 22:47:25.369465 17581 ts_tablet_manager.cc:1428] T 1949bad265884b36b5dec4d0842d0964 P a712a3a80999468ea0a596422fb8f3a1: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:25.369467 17576 consensus_queue.cc:260] T 1949bad265884b36b5dec4d0842d0964 P 4091dc731c9a4e15b38508539576d38e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a3c36e213304396bf7ebddea9300695" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 38371 } } peers { permanent_uuid: "a712a3a80999468ea0a596422fb8f3a1" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 44463 } } peers { permanent_uuid: "4091dc731c9a4e15b38508539576d38e" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45929 } }
I20260806 22:47:25.369557 17576 ts_tablet_manager.cc:1428] T 1949bad265884b36b5dec4d0842d0964 P 4091dc731c9a4e15b38508539576d38e: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260806 22:47:25.370209 17577 tablet_bootstrap.cc:492] T 1949bad265884b36b5dec4d0842d0964 P 1a3c36e213304396bf7ebddea9300695: No bootstrap required, opened a new log
I20260806 22:47:25.370258 17577 ts_tablet_manager.cc:1397] T 1949bad265884b36b5dec4d0842d0964 P 1a3c36e213304396bf7ebddea9300695: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260806 22:47:25.370393 17577 raft_consensus.cc:348] T 1949bad265884b36b5dec4d0842d0964 P 1a3c36e213304396bf7ebddea9300695 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a3c36e213304396bf7ebddea9300695" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 38371 } } peers { permanent_uuid: "a712a3a80999468ea0a596422fb8f3a1" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 44463 } } peers { permanent_uuid: "4091dc731c9a4e15b38508539576d38e" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45929 } }
I20260806 22:47:25.370455 17577 raft_consensus.cc:374] T 1949bad265884b36b5dec4d0842d0964 P 1a3c36e213304396bf7ebddea9300695 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:47:25.370476 17577 raft_consensus.cc:729] T 1949bad265884b36b5dec4d0842d0964 P 1a3c36e213304396bf7ebddea9300695 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a3c36e213304396bf7ebddea9300695, State: Initialized, Role: FOLLOWER
I20260806 22:47:25.370522 17577 consensus_queue.cc:260] T 1949bad265884b36b5dec4d0842d0964 P 1a3c36e213304396bf7ebddea9300695 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a3c36e213304396bf7ebddea9300695" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 38371 } } peers { permanent_uuid: "a712a3a80999468ea0a596422fb8f3a1" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 44463 } } peers { permanent_uuid: "4091dc731c9a4e15b38508539576d38e" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45929 } }
I20260806 22:47:25.370615 17577 ts_tablet_manager.cc:1428] T 1949bad265884b36b5dec4d0842d0964 P 1a3c36e213304396bf7ebddea9300695: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:25.665724 17634 raft_consensus.cc:482] T 1949bad265884b36b5dec4d0842d0964 P 4091dc731c9a4e15b38508539576d38e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:47:25.665895 17634 raft_consensus.cc:504] T 1949bad265884b36b5dec4d0842d0964 P 4091dc731c9a4e15b38508539576d38e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a3c36e213304396bf7ebddea9300695" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 38371 } } peers { permanent_uuid: "a712a3a80999468ea0a596422fb8f3a1" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 44463 } } peers { permanent_uuid: "4091dc731c9a4e15b38508539576d38e" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45929 } }
I20260806 22:47:25.666142 17634 leader_election.cc:288] T 1949bad265884b36b5dec4d0842d0964 P 4091dc731c9a4e15b38508539576d38e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1a3c36e213304396bf7ebddea9300695 (127.6.40.65:38371), a712a3a80999468ea0a596422fb8f3a1 (127.6.40.66:44463)
I20260806 22:47:25.666435 17286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1949bad265884b36b5dec4d0842d0964" candidate_uuid: "4091dc731c9a4e15b38508539576d38e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a712a3a80999468ea0a596422fb8f3a1" is_pre_election: true
I20260806 22:47:25.666435 17057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1949bad265884b36b5dec4d0842d0964" candidate_uuid: "4091dc731c9a4e15b38508539576d38e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a3c36e213304396bf7ebddea9300695" is_pre_election: true
I20260806 22:47:25.666561 17057 raft_consensus.cc:2454] T 1949bad265884b36b5dec4d0842d0964 P 1a3c36e213304396bf7ebddea9300695 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4091dc731c9a4e15b38508539576d38e in term 0.
I20260806 22:47:25.666561 17286 raft_consensus.cc:2454] T 1949bad265884b36b5dec4d0842d0964 P a712a3a80999468ea0a596422fb8f3a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4091dc731c9a4e15b38508539576d38e in term 0.
I20260806 22:47:25.666774 17385 leader_election.cc:302] T 1949bad265884b36b5dec4d0842d0964 P 4091dc731c9a4e15b38508539576d38e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4091dc731c9a4e15b38508539576d38e, a712a3a80999468ea0a596422fb8f3a1; no voters:
I20260806 22:47:25.666946 17634 raft_consensus.cc:2781] T 1949bad265884b36b5dec4d0842d0964 P 4091dc731c9a4e15b38508539576d38e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:47:25.667013 17634 raft_consensus.cc:482] T 1949bad265884b36b5dec4d0842d0964 P 4091dc731c9a4e15b38508539576d38e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:47:25.667042 17634 raft_consensus.cc:3037] T 1949bad265884b36b5dec4d0842d0964 P 4091dc731c9a4e15b38508539576d38e [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:47:25.668509 17634 raft_consensus.cc:504] T 1949bad265884b36b5dec4d0842d0964 P 4091dc731c9a4e15b38508539576d38e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a3c36e213304396bf7ebddea9300695" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 38371 } } peers { permanent_uuid: "a712a3a80999468ea0a596422fb8f3a1" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 44463 } } peers { permanent_uuid: "4091dc731c9a4e15b38508539576d38e" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45929 } }
I20260806 22:47:25.668699 17634 leader_election.cc:288] T 1949bad265884b36b5dec4d0842d0964 P 4091dc731c9a4e15b38508539576d38e [CANDIDATE]: Term 1 election: Requested vote from peers 1a3c36e213304396bf7ebddea9300695 (127.6.40.65:38371), a712a3a80999468ea0a596422fb8f3a1 (127.6.40.66:44463)
I20260806 22:47:25.668926 17057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1949bad265884b36b5dec4d0842d0964" candidate_uuid: "4091dc731c9a4e15b38508539576d38e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a3c36e213304396bf7ebddea9300695"
I20260806 22:47:25.669020 17286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1949bad265884b36b5dec4d0842d0964" candidate_uuid: "4091dc731c9a4e15b38508539576d38e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a712a3a80999468ea0a596422fb8f3a1"
I20260806 22:47:25.669086 17057 raft_consensus.cc:3037] T 1949bad265884b36b5dec4d0842d0964 P 1a3c36e213304396bf7ebddea9300695 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:47:25.669106 17286 raft_consensus.cc:3037] T 1949bad265884b36b5dec4d0842d0964 P a712a3a80999468ea0a596422fb8f3a1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:47:25.670009 17286 raft_consensus.cc:2454] T 1949bad265884b36b5dec4d0842d0964 P a712a3a80999468ea0a596422fb8f3a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4091dc731c9a4e15b38508539576d38e in term 1.
I20260806 22:47:25.670009 17057 raft_consensus.cc:2454] T 1949bad265884b36b5dec4d0842d0964 P 1a3c36e213304396bf7ebddea9300695 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4091dc731c9a4e15b38508539576d38e in term 1.
I20260806 22:47:25.670218 17385 leader_election.cc:302] T 1949bad265884b36b5dec4d0842d0964 P 4091dc731c9a4e15b38508539576d38e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4091dc731c9a4e15b38508539576d38e, a712a3a80999468ea0a596422fb8f3a1; no voters:
I20260806 22:47:25.670336 17634 raft_consensus.cc:2781] T 1949bad265884b36b5dec4d0842d0964 P 4091dc731c9a4e15b38508539576d38e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:47:25.670384 17634 raft_consensus.cc:686] T 1949bad265884b36b5dec4d0842d0964 P 4091dc731c9a4e15b38508539576d38e [term 1 LEADER]: Becoming Leader. State: Replica: 4091dc731c9a4e15b38508539576d38e, State: Running, Role: LEADER
I20260806 22:47:25.670441 17634 consensus_queue.cc:237] T 1949bad265884b36b5dec4d0842d0964 P 4091dc731c9a4e15b38508539576d38e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a3c36e213304396bf7ebddea9300695" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 38371 } } peers { permanent_uuid: "a712a3a80999468ea0a596422fb8f3a1" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 44463 } } peers { permanent_uuid: "4091dc731c9a4e15b38508539576d38e" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45929 } }
I20260806 22:47:25.671099 16858 catalog_manager.cc:5495] T 1949bad265884b36b5dec4d0842d0964 P 4091dc731c9a4e15b38508539576d38e reported cstate change: term changed from 0 to 1, leader changed from <none> to 4091dc731c9a4e15b38508539576d38e (127.6.40.67). New cstate: current_term: 1 leader_uuid: "4091dc731c9a4e15b38508539576d38e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a3c36e213304396bf7ebddea9300695" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 38371 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a712a3a80999468ea0a596422fb8f3a1" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 44463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4091dc731c9a4e15b38508539576d38e" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45929 } health_report { overall_health: HEALTHY } } }
I20260806 22:47:25.724167 17574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:25.721975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40006 (local address 127.6.40.67:45929)
0806 22:47:25.722066 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:25.722069 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:25.722080 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:47:25.722111 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:25.722112 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:25.722125 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:25.722173 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:47:25.722337 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:25.722859 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:25.723632 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:25.723809 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:25.723896 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:25.723995 (+ 99us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:47:25.724021 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:47:25.724057 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:25.724088 (+ 31us) server_negotiation.cc:299] Negotiation successful
0806 22:47:25.724107 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 22:47:25.725270 17286 raft_consensus.cc:1261] T 1949bad265884b36b5dec4d0842d0964 P a712a3a80999468ea0a596422fb8f3a1 [term 1 FOLLOWER]: Refusing update from remote peer 4091dc731c9a4e15b38508539576d38e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:47:25.725275 17057 raft_consensus.cc:1261] T 1949bad265884b36b5dec4d0842d0964 P 1a3c36e213304396bf7ebddea9300695 [term 1 FOLLOWER]: Refusing update from remote peer 4091dc731c9a4e15b38508539576d38e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:47:25.725553 17586 consensus_queue.cc:1035] T 1949bad265884b36b5dec4d0842d0964 P 4091dc731c9a4e15b38508539576d38e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a3c36e213304396bf7ebddea9300695" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 38371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:47:25.725663 17634 consensus_queue.cc:1035] T 1949bad265884b36b5dec4d0842d0964 P 4091dc731c9a4e15b38508539576d38e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a712a3a80999468ea0a596422fb8f3a1" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:47:25.729519 16858 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59704:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:47:25.729619 16858 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59704:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:47:25.730306 16858 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a023cd9df99d4e3db64a1b4977884980: Sending DeleteTablet for 3 replicas of tablet 1949bad265884b36b5dec4d0842d0964
I20260806 22:47:25.730607 17052 tablet_service.cc:1514] Processing DeleteTablet for tablet 1949bad265884b36b5dec4d0842d0964 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:47:25 UTC) from {username='oryx', principal='oryx/127.6.40.126@KRBTEST.COM'} at 127.0.0.1:35748
I20260806 22:47:25.730595 17257 tablet_service.cc:1514] Processing DeleteTablet for tablet 1949bad265884b36b5dec4d0842d0964 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:47:25 UTC) from {username='oryx', principal='oryx/127.6.40.126@KRBTEST.COM'} at 127.0.0.1:57432
I20260806 22:47:25.730703 17611 tablet_replica.cc:330] T 1949bad265884b36b5dec4d0842d0964 P 1a3c36e213304396bf7ebddea9300695: stopping tablet replica
I20260806 22:47:25.730723 17609 tablet_replica.cc:330] T 1949bad265884b36b5dec4d0842d0964 P a712a3a80999468ea0a596422fb8f3a1: stopping tablet replica
I20260806 22:47:25.730754 17611 raft_consensus.cc:2229] T 1949bad265884b36b5dec4d0842d0964 P 1a3c36e213304396bf7ebddea9300695 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:47:25.730785 17609 raft_consensus.cc:2229] T 1949bad265884b36b5dec4d0842d0964 P a712a3a80999468ea0a596422fb8f3a1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:47:25.730795 17611 raft_consensus.cc:2258] T 1949bad265884b36b5dec4d0842d0964 P 1a3c36e213304396bf7ebddea9300695 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:47:25.730831 17609 raft_consensus.cc:2258] T 1949bad265884b36b5dec4d0842d0964 P a712a3a80999468ea0a596422fb8f3a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:47:25.731189 17609 ts_tablet_manager.cc:1905] T 1949bad265884b36b5dec4d0842d0964 P a712a3a80999468ea0a596422fb8f3a1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:47:25.731253 17611 ts_tablet_manager.cc:1905] T 1949bad265884b36b5dec4d0842d0964 P 1a3c36e213304396bf7ebddea9300695: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:47:25.732632 17609 ts_tablet_manager.cc:1918] T 1949bad265884b36b5dec4d0842d0964 P a712a3a80999468ea0a596422fb8f3a1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:47:25.732709 17609 log.cc:1196] T 1949bad265884b36b5dec4d0842d0964 P a712a3a80999468ea0a596422fb8f3a1: Deleting WAL directory at /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/minicluster-data/ts-1/wal/wals/1949bad265884b36b5dec4d0842d0964
I20260806 22:47:25.732892 17611 ts_tablet_manager.cc:1918] T 1949bad265884b36b5dec4d0842d0964 P 1a3c36e213304396bf7ebddea9300695: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:47:25.732954 17611 log.cc:1196] T 1949bad265884b36b5dec4d0842d0964 P 1a3c36e213304396bf7ebddea9300695: Deleting WAL directory at /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/minicluster-data/ts-0/wal/wals/1949bad265884b36b5dec4d0842d0964
I20260806 22:47:25.732998 17609 ts_tablet_manager.cc:1939] T 1949bad265884b36b5dec4d0842d0964 P a712a3a80999468ea0a596422fb8f3a1: Deleting consensus metadata
I20260806 22:47:25.733207 17611 ts_tablet_manager.cc:1939] T 1949bad265884b36b5dec4d0842d0964 P 1a3c36e213304396bf7ebddea9300695: Deleting consensus metadata
I20260806 22:47:25.733469 17450 tablet_service.cc:1514] Processing DeleteTablet for tablet 1949bad265884b36b5dec4d0842d0964 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:47:25 UTC) from {username='oryx', principal='oryx/127.6.40.126@KRBTEST.COM'} at 127.0.0.1:37116
I20260806 22:47:25.733587 17610 tablet_replica.cc:330] T 1949bad265884b36b5dec4d0842d0964 P 4091dc731c9a4e15b38508539576d38e: stopping tablet replica
I20260806 22:47:25.733656 17610 raft_consensus.cc:2229] T 1949bad265884b36b5dec4d0842d0964 P 4091dc731c9a4e15b38508539576d38e [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:47:25.733744 17610 raft_consensus.cc:2258] T 1949bad265884b36b5dec4d0842d0964 P 4091dc731c9a4e15b38508539576d38e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:47:25.733928 16834 catalog_manager.cc:4841] TS 1a3c36e213304396bf7ebddea9300695 (127.6.40.65:38371): tablet 1949bad265884b36b5dec4d0842d0964 (table test-table [id=64d259b6d28242ba829f5957beb81244]) successfully deleted
I20260806 22:47:25.734122 17610 ts_tablet_manager.cc:1905] T 1949bad265884b36b5dec4d0842d0964 P 4091dc731c9a4e15b38508539576d38e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:47:25.735582 17610 ts_tablet_manager.cc:1918] T 1949bad265884b36b5dec4d0842d0964 P 4091dc731c9a4e15b38508539576d38e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:47:25.735647 17610 log.cc:1196] T 1949bad265884b36b5dec4d0842d0964 P 4091dc731c9a4e15b38508539576d38e: Deleting WAL directory at /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/minicluster-data/ts-2/wal/wals/1949bad265884b36b5dec4d0842d0964
I20260806 22:47:25.735873 17610 ts_tablet_manager.cc:1939] T 1949bad265884b36b5dec4d0842d0964 P 4091dc731c9a4e15b38508539576d38e: Deleting consensus metadata
I20260806 22:47:25.736272 16838 catalog_manager.cc:4841] TS 4091dc731c9a4e15b38508539576d38e (127.6.40.67:45929): tablet 1949bad265884b36b5dec4d0842d0964 (table test-table [id=64d259b6d28242ba829f5957beb81244]) successfully deleted
I20260806 22:47:25.736340 16838 catalog_manager.cc:4841] TS a712a3a80999468ea0a596422fb8f3a1 (127.6.40.66:44463): tablet 1949bad265884b36b5dec4d0842d0964 (table test-table [id=64d259b6d28242ba829f5957beb81244]) successfully deleted
I20260806 22:47:25.736299 16858 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59704:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:47:25.736567 16858 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:47:25.737824 17052 tablet_service.cc:1467] Processing CreateTablet for tablet 2a7e58a73d48452d8156cd63e5daa1fc (DEFAULT_TABLE table=test-table [id=df71d02c4a4844f89e19a5429933d4d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:47:25.737895 17257 tablet_service.cc:1467] Processing CreateTablet for tablet 2a7e58a73d48452d8156cd63e5daa1fc (DEFAULT_TABLE table=test-table [id=df71d02c4a4844f89e19a5429933d4d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:47:25.737965 17052 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a7e58a73d48452d8156cd63e5daa1fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:47:25.738010 17257 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a7e58a73d48452d8156cd63e5daa1fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:47:25.739260 17581 tablet_bootstrap.cc:492] T 2a7e58a73d48452d8156cd63e5daa1fc P a712a3a80999468ea0a596422fb8f3a1: Bootstrap starting.
I20260806 22:47:25.739305 17577 tablet_bootstrap.cc:492] T 2a7e58a73d48452d8156cd63e5daa1fc P 1a3c36e213304396bf7ebddea9300695: Bootstrap starting.
I20260806 22:47:25.739602 17581 tablet_bootstrap.cc:654] T 2a7e58a73d48452d8156cd63e5daa1fc P a712a3a80999468ea0a596422fb8f3a1: Neither blocks nor log segments found. Creating new log.
I20260806 22:47:25.739632 17577 tablet_bootstrap.cc:654] T 2a7e58a73d48452d8156cd63e5daa1fc P 1a3c36e213304396bf7ebddea9300695: Neither blocks nor log segments found. Creating new log.
I20260806 22:47:25.740036 17450 tablet_service.cc:1467] Processing CreateTablet for tablet 2a7e58a73d48452d8156cd63e5daa1fc (DEFAULT_TABLE table=test-table [id=df71d02c4a4844f89e19a5429933d4d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:47:25.740150 17450 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a7e58a73d48452d8156cd63e5daa1fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:47:25.740195 17581 tablet_bootstrap.cc:492] T 2a7e58a73d48452d8156cd63e5daa1fc P a712a3a80999468ea0a596422fb8f3a1: No bootstrap required, opened a new log
I20260806 22:47:25.740247 17581 ts_tablet_manager.cc:1397] T 2a7e58a73d48452d8156cd63e5daa1fc P a712a3a80999468ea0a596422fb8f3a1: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260806 22:47:25.740366 17581 raft_consensus.cc:348] T 2a7e58a73d48452d8156cd63e5daa1fc P a712a3a80999468ea0a596422fb8f3a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a3c36e213304396bf7ebddea9300695" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 38371 } } peers { permanent_uuid: "a712a3a80999468ea0a596422fb8f3a1" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 44463 } } peers { permanent_uuid: "4091dc731c9a4e15b38508539576d38e" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45929 } }
I20260806 22:47:25.740459 17581 raft_consensus.cc:374] T 2a7e58a73d48452d8156cd63e5daa1fc P a712a3a80999468ea0a596422fb8f3a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:47:25.740489 17581 raft_consensus.cc:729] T 2a7e58a73d48452d8156cd63e5daa1fc P a712a3a80999468ea0a596422fb8f3a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a712a3a80999468ea0a596422fb8f3a1, State: Initialized, Role: FOLLOWER
I20260806 22:47:25.740532 17581 consensus_queue.cc:260] T 2a7e58a73d48452d8156cd63e5daa1fc P a712a3a80999468ea0a596422fb8f3a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a3c36e213304396bf7ebddea9300695" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 38371 } } peers { permanent_uuid: "a712a3a80999468ea0a596422fb8f3a1" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 44463 } } peers { permanent_uuid: "4091dc731c9a4e15b38508539576d38e" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45929 } }
I20260806 22:47:25.740633 17581 ts_tablet_manager.cc:1428] T 2a7e58a73d48452d8156cd63e5daa1fc P a712a3a80999468ea0a596422fb8f3a1: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260806 22:47:25.741371 17576 tablet_bootstrap.cc:492] T 2a7e58a73d48452d8156cd63e5daa1fc P 4091dc731c9a4e15b38508539576d38e: Bootstrap starting.
I20260806 22:47:25.741545 17577 tablet_bootstrap.cc:492] T 2a7e58a73d48452d8156cd63e5daa1fc P 1a3c36e213304396bf7ebddea9300695: No bootstrap required, opened a new log
I20260806 22:47:25.741602 17577 ts_tablet_manager.cc:1397] T 2a7e58a73d48452d8156cd63e5daa1fc P 1a3c36e213304396bf7ebddea9300695: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:47:25.741708 17577 raft_consensus.cc:348] T 2a7e58a73d48452d8156cd63e5daa1fc P 1a3c36e213304396bf7ebddea9300695 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a3c36e213304396bf7ebddea9300695" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 38371 } } peers { permanent_uuid: "a712a3a80999468ea0a596422fb8f3a1" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 44463 } } peers { permanent_uuid: "4091dc731c9a4e15b38508539576d38e" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45929 } }
I20260806 22:47:25.741780 17577 raft_consensus.cc:374] T 2a7e58a73d48452d8156cd63e5daa1fc P 1a3c36e213304396bf7ebddea9300695 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:47:25.741801 17577 raft_consensus.cc:729] T 2a7e58a73d48452d8156cd63e5daa1fc P 1a3c36e213304396bf7ebddea9300695 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a3c36e213304396bf7ebddea9300695, State: Initialized, Role: FOLLOWER
I20260806 22:47:25.741822 17576 tablet_bootstrap.cc:654] T 2a7e58a73d48452d8156cd63e5daa1fc P 4091dc731c9a4e15b38508539576d38e: Neither blocks nor log segments found. Creating new log.
I20260806 22:47:25.741850 17577 consensus_queue.cc:260] T 2a7e58a73d48452d8156cd63e5daa1fc P 1a3c36e213304396bf7ebddea9300695 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a3c36e213304396bf7ebddea9300695" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 38371 } } peers { permanent_uuid: "a712a3a80999468ea0a596422fb8f3a1" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 44463 } } peers { permanent_uuid: "4091dc731c9a4e15b38508539576d38e" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45929 } }
I20260806 22:47:25.741933 17577 ts_tablet_manager.cc:1428] T 2a7e58a73d48452d8156cd63e5daa1fc P 1a3c36e213304396bf7ebddea9300695: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 22:47:25.742406 17576 tablet_bootstrap.cc:492] T 2a7e58a73d48452d8156cd63e5daa1fc P 4091dc731c9a4e15b38508539576d38e: No bootstrap required, opened a new log
I20260806 22:47:25.742475 17576 ts_tablet_manager.cc:1397] T 2a7e58a73d48452d8156cd63e5daa1fc P 4091dc731c9a4e15b38508539576d38e: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260806 22:47:25.742605 17576 raft_consensus.cc:348] T 2a7e58a73d48452d8156cd63e5daa1fc P 4091dc731c9a4e15b38508539576d38e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a3c36e213304396bf7ebddea9300695" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 38371 } } peers { permanent_uuid: "a712a3a80999468ea0a596422fb8f3a1" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 44463 } } peers { permanent_uuid: "4091dc731c9a4e15b38508539576d38e" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45929 } }
I20260806 22:47:25.742689 17576 raft_consensus.cc:374] T 2a7e58a73d48452d8156cd63e5daa1fc P 4091dc731c9a4e15b38508539576d38e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:47:25.742714 17576 raft_consensus.cc:729] T 2a7e58a73d48452d8156cd63e5daa1fc P 4091dc731c9a4e15b38508539576d38e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4091dc731c9a4e15b38508539576d38e, State: Initialized, Role: FOLLOWER
I20260806 22:47:25.742762 17576 consensus_queue.cc:260] T 2a7e58a73d48452d8156cd63e5daa1fc P 4091dc731c9a4e15b38508539576d38e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a3c36e213304396bf7ebddea9300695" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 38371 } } peers { permanent_uuid: "a712a3a80999468ea0a596422fb8f3a1" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 44463 } } peers { permanent_uuid: "4091dc731c9a4e15b38508539576d38e" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45929 } }
I20260806 22:47:25.742846 17576 ts_tablet_manager.cc:1428] T 2a7e58a73d48452d8156cd63e5daa1fc P 4091dc731c9a4e15b38508539576d38e: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260806 22:47:25.778424 17586 raft_consensus.cc:482] T 2a7e58a73d48452d8156cd63e5daa1fc P 4091dc731c9a4e15b38508539576d38e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:47:25.778553 17586 raft_consensus.cc:504] T 2a7e58a73d48452d8156cd63e5daa1fc P 4091dc731c9a4e15b38508539576d38e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a3c36e213304396bf7ebddea9300695" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 38371 } } peers { permanent_uuid: "a712a3a80999468ea0a596422fb8f3a1" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 44463 } } peers { permanent_uuid: "4091dc731c9a4e15b38508539576d38e" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45929 } }
I20260806 22:47:25.778753 17586 leader_election.cc:288] T 2a7e58a73d48452d8156cd63e5daa1fc P 4091dc731c9a4e15b38508539576d38e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1a3c36e213304396bf7ebddea9300695 (127.6.40.65:38371), a712a3a80999468ea0a596422fb8f3a1 (127.6.40.66:44463)
I20260806 22:47:25.778988 17057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a7e58a73d48452d8156cd63e5daa1fc" candidate_uuid: "4091dc731c9a4e15b38508539576d38e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a3c36e213304396bf7ebddea9300695" is_pre_election: true
I20260806 22:47:25.778987 17286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a7e58a73d48452d8156cd63e5daa1fc" candidate_uuid: "4091dc731c9a4e15b38508539576d38e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a712a3a80999468ea0a596422fb8f3a1" is_pre_election: true
I20260806 22:47:25.779101 17057 raft_consensus.cc:2454] T 2a7e58a73d48452d8156cd63e5daa1fc P 1a3c36e213304396bf7ebddea9300695 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4091dc731c9a4e15b38508539576d38e in term 0.
I20260806 22:47:25.779101 17286 raft_consensus.cc:2454] T 2a7e58a73d48452d8156cd63e5daa1fc P a712a3a80999468ea0a596422fb8f3a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4091dc731c9a4e15b38508539576d38e in term 0.
I20260806 22:47:25.779273 17385 leader_election.cc:302] T 2a7e58a73d48452d8156cd63e5daa1fc P 4091dc731c9a4e15b38508539576d38e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4091dc731c9a4e15b38508539576d38e, a712a3a80999468ea0a596422fb8f3a1; no voters:
I20260806 22:47:25.779448 17586 raft_consensus.cc:2781] T 2a7e58a73d48452d8156cd63e5daa1fc P 4091dc731c9a4e15b38508539576d38e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:47:25.779527 17586 raft_consensus.cc:482] T 2a7e58a73d48452d8156cd63e5daa1fc P 4091dc731c9a4e15b38508539576d38e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:47:25.779570 17586 raft_consensus.cc:3037] T 2a7e58a73d48452d8156cd63e5daa1fc P 4091dc731c9a4e15b38508539576d38e [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:47:25.780267 17586 raft_consensus.cc:504] T 2a7e58a73d48452d8156cd63e5daa1fc P 4091dc731c9a4e15b38508539576d38e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a3c36e213304396bf7ebddea9300695" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 38371 } } peers { permanent_uuid: "a712a3a80999468ea0a596422fb8f3a1" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 44463 } } peers { permanent_uuid: "4091dc731c9a4e15b38508539576d38e" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45929 } }
I20260806 22:47:25.780416 17586 leader_election.cc:288] T 2a7e58a73d48452d8156cd63e5daa1fc P 4091dc731c9a4e15b38508539576d38e [CANDIDATE]: Term 1 election: Requested vote from peers 1a3c36e213304396bf7ebddea9300695 (127.6.40.65:38371), a712a3a80999468ea0a596422fb8f3a1 (127.6.40.66:44463)
I20260806 22:47:25.780560 17286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a7e58a73d48452d8156cd63e5daa1fc" candidate_uuid: "4091dc731c9a4e15b38508539576d38e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a712a3a80999468ea0a596422fb8f3a1"
I20260806 22:47:25.780588 17057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a7e58a73d48452d8156cd63e5daa1fc" candidate_uuid: "4091dc731c9a4e15b38508539576d38e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a3c36e213304396bf7ebddea9300695"
I20260806 22:47:25.780647 17286 raft_consensus.cc:3037] T 2a7e58a73d48452d8156cd63e5daa1fc P a712a3a80999468ea0a596422fb8f3a1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:47:25.780676 17057 raft_consensus.cc:3037] T 2a7e58a73d48452d8156cd63e5daa1fc P 1a3c36e213304396bf7ebddea9300695 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:47:25.781487 17057 raft_consensus.cc:2454] T 2a7e58a73d48452d8156cd63e5daa1fc P 1a3c36e213304396bf7ebddea9300695 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4091dc731c9a4e15b38508539576d38e in term 1.
I20260806 22:47:25.781543 17286 raft_consensus.cc:2454] T 2a7e58a73d48452d8156cd63e5daa1fc P a712a3a80999468ea0a596422fb8f3a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4091dc731c9a4e15b38508539576d38e in term 1.
I20260806 22:47:25.781714 17385 leader_election.cc:302] T 2a7e58a73d48452d8156cd63e5daa1fc P 4091dc731c9a4e15b38508539576d38e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4091dc731c9a4e15b38508539576d38e, a712a3a80999468ea0a596422fb8f3a1; no voters:
I20260806 22:47:25.781819 17634 raft_consensus.cc:2781] T 2a7e58a73d48452d8156cd63e5daa1fc P 4091dc731c9a4e15b38508539576d38e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:47:25.781901 17634 raft_consensus.cc:686] T 2a7e58a73d48452d8156cd63e5daa1fc P 4091dc731c9a4e15b38508539576d38e [term 1 LEADER]: Becoming Leader. State: Replica: 4091dc731c9a4e15b38508539576d38e, State: Running, Role: LEADER
I20260806 22:47:25.781944 17634 consensus_queue.cc:237] T 2a7e58a73d48452d8156cd63e5daa1fc P 4091dc731c9a4e15b38508539576d38e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a3c36e213304396bf7ebddea9300695" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 38371 } } peers { permanent_uuid: "a712a3a80999468ea0a596422fb8f3a1" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 44463 } } peers { permanent_uuid: "4091dc731c9a4e15b38508539576d38e" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45929 } }
I20260806 22:47:25.782498 16858 catalog_manager.cc:5495] T 2a7e58a73d48452d8156cd63e5daa1fc P 4091dc731c9a4e15b38508539576d38e reported cstate change: term changed from 0 to 1, leader changed from <none> to 4091dc731c9a4e15b38508539576d38e (127.6.40.67). New cstate: current_term: 1 leader_uuid: "4091dc731c9a4e15b38508539576d38e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a3c36e213304396bf7ebddea9300695" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 38371 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a712a3a80999468ea0a596422fb8f3a1" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 44463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4091dc731c9a4e15b38508539576d38e" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 45929 } health_report { overall_health: HEALTHY } } }
I20260806 22:47:25.796521 17574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:25.794120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.67:48159 (local address 127.6.40.67:45929)
0806 22:47:25.794150 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:25.794152 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:25.794156 (+ 4us) server_negotiation.cc:407] Connection header received
0806 22:47:25.794169 (+ 13us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:25.794169 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:25.794176 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:25.794232 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:47:25.794361 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:25.794833 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:25.795736 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:25.795942 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:25.796030 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:25.796194 (+ 164us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:47:25.796232 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:47:25.796294 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:25.796414 (+ 120us) server_negotiation.cc:299] Negotiation successful
0806 22:47:25.796431 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":19}
I20260806 22:47:25.796521 17653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:25.793973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.67:45929 (local address 127.6.40.67:48159)
0806 22:47:25.794029 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:25.794040 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:25.794084 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:25.794236 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:25.794237 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:25.794238 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 22:47:25.794294 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:25.794295 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:25.794914 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:25.794915 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:25.795577 (+ 662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:25.795580 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:25.796031 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:25.796032 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:25.796108 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:25.796114 (+ 6us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 22:47:25.796294 (+ 180us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 22:47:25.796296 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 22:47:25.796390 (+ 94us) client_negotiation.cc:240] Negotiation successful
0806 22:47:25.796437 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31,"mutex_wait_us":3}
I20260806 22:47:25.798645 17286 raft_consensus.cc:1261] T 2a7e58a73d48452d8156cd63e5daa1fc P a712a3a80999468ea0a596422fb8f3a1 [term 1 FOLLOWER]: Refusing update from remote peer 4091dc731c9a4e15b38508539576d38e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:47:25.798646 17057 raft_consensus.cc:1261] T 2a7e58a73d48452d8156cd63e5daa1fc P 1a3c36e213304396bf7ebddea9300695 [term 1 FOLLOWER]: Refusing update from remote peer 4091dc731c9a4e15b38508539576d38e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:47:25.798893 17634 consensus_queue.cc:1035] T 2a7e58a73d48452d8156cd63e5daa1fc P 4091dc731c9a4e15b38508539576d38e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a712a3a80999468ea0a596422fb8f3a1" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:47:25.799005 17586 consensus_queue.cc:1035] T 2a7e58a73d48452d8156cd63e5daa1fc P 4091dc731c9a4e15b38508539576d38e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a3c36e213304396bf7ebddea9300695" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 38371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:47:25.809551 16858 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59704:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:47:25.809669 16858 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59704:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:47:25.810374 16858 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a023cd9df99d4e3db64a1b4977884980: Sending DeleteTablet for 3 replicas of tablet 2a7e58a73d48452d8156cd63e5daa1fc
I20260806 22:47:25.810657 17052 tablet_service.cc:1514] Processing DeleteTablet for tablet 2a7e58a73d48452d8156cd63e5daa1fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:47:25 UTC) from {username='oryx', principal='oryx/127.6.40.126@KRBTEST.COM'} at 127.0.0.1:35748
I20260806 22:47:25.810724 17257 tablet_service.cc:1514] Processing DeleteTablet for tablet 2a7e58a73d48452d8156cd63e5daa1fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:47:25 UTC) from {username='oryx', principal='oryx/127.6.40.126@KRBTEST.COM'} at 127.0.0.1:57432
I20260806 22:47:25.810787 17611 tablet_replica.cc:330] T 2a7e58a73d48452d8156cd63e5daa1fc P 1a3c36e213304396bf7ebddea9300695: stopping tablet replica
I20260806 22:47:25.810868 17611 raft_consensus.cc:2229] T 2a7e58a73d48452d8156cd63e5daa1fc P 1a3c36e213304396bf7ebddea9300695 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:47:25.810845 17450 tablet_service.cc:1514] Processing DeleteTablet for tablet 2a7e58a73d48452d8156cd63e5daa1fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:47:25 UTC) from {username='oryx', principal='oryx/127.6.40.126@KRBTEST.COM'} at 127.0.0.1:37116
I20260806 22:47:25.810932 17611 raft_consensus.cc:2258] T 2a7e58a73d48452d8156cd63e5daa1fc P 1a3c36e213304396bf7ebddea9300695 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:47:25.810956 17610 tablet_replica.cc:330] T 2a7e58a73d48452d8156cd63e5daa1fc P 4091dc731c9a4e15b38508539576d38e: stopping tablet replica
I20260806 22:47:25.811009 17610 raft_consensus.cc:2229] T 2a7e58a73d48452d8156cd63e5daa1fc P 4091dc731c9a4e15b38508539576d38e [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:47:25.811017 17609 tablet_replica.cc:330] T 2a7e58a73d48452d8156cd63e5daa1fc P a712a3a80999468ea0a596422fb8f3a1: stopping tablet replica
I20260806 22:47:25.811072 17610 raft_consensus.cc:2258] T 2a7e58a73d48452d8156cd63e5daa1fc P 4091dc731c9a4e15b38508539576d38e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:47:25.811079 17609 raft_consensus.cc:2229] T 2a7e58a73d48452d8156cd63e5daa1fc P a712a3a80999468ea0a596422fb8f3a1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:47:25.811118 17609 raft_consensus.cc:2258] T 2a7e58a73d48452d8156cd63e5daa1fc P a712a3a80999468ea0a596422fb8f3a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:47:25.811594 17609 ts_tablet_manager.cc:1905] T 2a7e58a73d48452d8156cd63e5daa1fc P a712a3a80999468ea0a596422fb8f3a1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:47:25.813010 17609 ts_tablet_manager.cc:1918] T 2a7e58a73d48452d8156cd63e5daa1fc P a712a3a80999468ea0a596422fb8f3a1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:47:25.813022 17611 ts_tablet_manager.cc:1905] T 2a7e58a73d48452d8156cd63e5daa1fc P 1a3c36e213304396bf7ebddea9300695: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:47:25.813086 17609 log.cc:1196] T 2a7e58a73d48452d8156cd63e5daa1fc P a712a3a80999468ea0a596422fb8f3a1: Deleting WAL directory at /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/minicluster-data/ts-1/wal/wals/2a7e58a73d48452d8156cd63e5daa1fc
I20260806 22:47:25.813197 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 16961
I20260806 22:47:25.813305 17609 ts_tablet_manager.cc:1939] T 2a7e58a73d48452d8156cd63e5daa1fc P a712a3a80999468ea0a596422fb8f3a1: Deleting consensus metadata
I20260806 22:47:25.816049 16838 catalog_manager.cc:4841] TS a712a3a80999468ea0a596422fb8f3a1 (127.6.40.66:44463): tablet 2a7e58a73d48452d8156cd63e5daa1fc (table test-table [id=df71d02c4a4844f89e19a5429933d4d3]) successfully deleted
I20260806 22:47:25.821231 17610 ts_tablet_manager.cc:1905] T 2a7e58a73d48452d8156cd63e5daa1fc P 4091dc731c9a4e15b38508539576d38e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:47:25.822952 17610 ts_tablet_manager.cc:1918] T 2a7e58a73d48452d8156cd63e5daa1fc P 4091dc731c9a4e15b38508539576d38e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:47:25.823026 17610 log.cc:1196] T 2a7e58a73d48452d8156cd63e5daa1fc P 4091dc731c9a4e15b38508539576d38e: Deleting WAL directory at /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056405724609-6305-0/minicluster-data/ts-2/wal/wals/2a7e58a73d48452d8156cd63e5daa1fc
I20260806 22:47:25.823252 17610 ts_tablet_manager.cc:1939] T 2a7e58a73d48452d8156cd63e5daa1fc P 4091dc731c9a4e15b38508539576d38e: Deleting consensus metadata
I20260806 22:47:25.823640 16838 catalog_manager.cc:4841] TS 4091dc731c9a4e15b38508539576d38e (127.6.40.67:45929): tablet 2a7e58a73d48452d8156cd63e5daa1fc (table test-table [id=df71d02c4a4844f89e19a5429933d4d3]) successfully deleted
W20260806 22:47:25.825690 16834 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.40.65:38371 (error 108)
I20260806 22:47:25.826285 17693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:25.826092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.65:38371 (local address 127.0.0.1:36460)
0806 22:47:25.826180 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:25.826243 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.65:38371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69,"thread_start_us":44,"threads_started":1}
I20260806 22:47:25.826390 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 17169
W20260806 22:47:25.826399 16834 catalog_manager.cc:4584] TS 1a3c36e213304396bf7ebddea9300695 (127.6.40.65:38371): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2a7e58a73d48452d8156cd63e5daa1fc: Network error: Client connection negotiation failed: client connection to 127.6.40.65:38371: connect: Connection refused (error 111)
I20260806 22:47:25.849357 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 17363
I20260806 22:47:25.861826 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 16810
2026-08-06T22:47:25Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3321 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786056405724609-6305-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:47:25 dist-test-slave-zh2d krb5kdc[17696](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:47:25 dist-test-slave-zh2d krb5kdc[17696](info): set up 2 sockets
Aug 06 22:47:25 dist-test-slave-zh2d krb5kdc[17696](info): commencing operation
krb5kdc: starting...
W20260806 22:47:27.094209 6305 mini_kdc.cc:121] Time spent starting KDC: real 1.207s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786056405724609-6305-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786056405724609-6305-0/krb5kdc/test-user.keytab.
Aug 06 22:47:27 dist-test-slave-zh2d krb5kdc[17696](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056447, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:47:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:47:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.40.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.40.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.40.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab.
Entry for principal kudu/127.6.40.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab.
I20260806 22:47:27.225078 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786056405724609-6305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786056405724609-6305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786056405724609-6305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.126:39703
--webserver_interface=127.6.40.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab
--principal=kudu/127.6.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:41463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.40.126:39703
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:47:27.309878 17714 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:47:27.310097 17714 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:47:27.310119 17714 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:47:27.311208 17714 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:47:27.311244 17714 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:47:27.311259 17714 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:47:27.311270 17714 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:47:27.311283 17714 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:47:27.312714 17714 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:41463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786056405724609-6305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786056405724609-6305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.40.126:39703
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.126:39703
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17714
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786056405724609-6305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:47:27.312968 17714 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:47:27.313174 17714 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:47:27.318188 17727 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:27.318152 17725 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:27.318205 17724 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:27.318743 17714 server_base.cc:1034] running on GCE node
I20260806 22:47:27.319062 17714 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:47:27.319382 17714 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:47:27.320529 17714 hybrid_clock.cc:648] HybridClock initialized: now 1786056447320507 us; error 38 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-06T22:47:27Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1467 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:47:27 dist-test-slave-zh2d krb5kdc[17732](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:47:27 dist-test-slave-zh2d krb5kdc[17732](info): set up 2 sockets
Aug 06 22:47:27 dist-test-slave-zh2d krb5kdc[17732](info): commencing operation
krb5kdc: starting...
W20260806 22:47:28.546093 6305 mini_kdc.cc:121] Time spent starting KDC: real 1.191s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/krb5kdc/test-user.keytab.
Aug 06 22:47:28 dist-test-slave-zh2d krb5kdc[17732](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056448, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:47:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:47:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.40.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.40.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.40.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab.
Entry for principal kudu/127.6.40.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab.
I20260806 22:47:28.672214 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.126:44911
--webserver_interface=127.6.40.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab
--principal=kudu/127.6.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:45855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.40.126:44911
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:47:28.757936 17749 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:47:28.758188 17749 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:47:28.758226 17749 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:47:28.759385 17749 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:47:28.759457 17749 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:47:28.759481 17749 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:47:28.759502 17749 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:47:28.759531 17749 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:47:28.761037 17749 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:45855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.40.126:44911
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab
--principal=kudu/127.6.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.126:44911
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17749
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:47:28.761322 17749 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:47:28.761556 17749 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:47:28.766660 17759 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:28.766686 17754 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:28.766705 17749 server_base.cc:1034] running on GCE node
W20260806 22:47:28.766686 17762 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:28.767159 17749 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:47:28.767402 17749 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:47:28.768539 17749 hybrid_clock.cc:648] HybridClock initialized: now 1786056448768530 us; error 34 us; skew 500 ppm
Aug 06 22:47:28 dist-test-slave-zh2d krb5kdc[17732](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056448, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:47:28.771183 17749 init.cc:377] Logged in from keytab as kudu/127.6.40.126@KRBTEST.COM (short username kudu)
I20260806 22:47:28.771626 17749 webserver.cc:460] Webserver started at http://127.6.40.126:46729/ using document root <none> and password file <none>
I20260806 22:47:28.771852 17749 fs_manager.cc:359] Metadata directory not provided
I20260806 22:47:28.771896 17749 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:47:28.772053 17749 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:47:28.773187 17749 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/master-0/data/instance:
uuid: "922f6e3a3609453f9f76730f68829f39"
format_stamp: "Formatted at 2026-08-06 22:47:28 on dist-test-slave-zh2d"
server_key: "12bcc3aa6829694d9fc36e61ec2c0164"
server_key_iv: "e805aff0d6521c77f34f8cbb52b7dbcd"
server_key_version: "encryptionkey@0"
I20260806 22:47:28.773555 17749 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/master-0/wal/instance:
uuid: "922f6e3a3609453f9f76730f68829f39"
format_stamp: "Formatted at 2026-08-06 22:47:28 on dist-test-slave-zh2d"
server_key: "12bcc3aa6829694d9fc36e61ec2c0164"
server_key_iv: "e805aff0d6521c77f34f8cbb52b7dbcd"
server_key_version: "encryptionkey@0"
I20260806 22:47:28.774973 17749 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:47:28.775895 17772 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:28.776165 17749 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:47:28.776288 17749 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/master-0/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/master-0/wal
uuid: "922f6e3a3609453f9f76730f68829f39"
format_stamp: "Formatted at 2026-08-06 22:47:28 on dist-test-slave-zh2d"
server_key: "12bcc3aa6829694d9fc36e61ec2c0164"
server_key_iv: "e805aff0d6521c77f34f8cbb52b7dbcd"
server_key_version: "encryptionkey@0"
I20260806 22:47:28.776377 17749 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:47:28.787442 17749 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:47:28.789645 17749 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:47:28.789813 17749 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:47:28.794281 17749 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.126:44911
I20260806 22:47:28.794481 17843 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.126:44911 every 8 connection(s)
I20260806 22:47:28.794725 17749 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
I20260806 22:47:28.795671 17844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:47:28.798089 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 17749
I20260806 22:47:28.798239 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/master-0/wal/instance
I20260806 22:47:28.798362 17844 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 922f6e3a3609453f9f76730f68829f39: Bootstrap starting.
I20260806 22:47:28.798441 6305 external_mini_cluster.cc:1428] Setting key 3896e98042034367b5e9444bc6062b4e
I20260806 22:47:28.799139 17844 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 922f6e3a3609453f9f76730f68829f39: Neither blocks nor log segments found. Creating new log.
I20260806 22:47:28.799486 17844 log.cc:824] T 00000000000000000000000000000000 P 922f6e3a3609453f9f76730f68829f39: Log is configured to *not* fsync() on all Append() calls
I20260806 22:47:28.800242 17844 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 922f6e3a3609453f9f76730f68829f39: No bootstrap required, opened a new log
I20260806 22:47:28.802368 17844 raft_consensus.cc:348] T 00000000000000000000000000000000 P 922f6e3a3609453f9f76730f68829f39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "922f6e3a3609453f9f76730f68829f39" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 44911 } }
I20260806 22:47:28.802532 17844 raft_consensus.cc:374] T 00000000000000000000000000000000 P 922f6e3a3609453f9f76730f68829f39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:47:28.802584 17844 raft_consensus.cc:729] T 00000000000000000000000000000000 P 922f6e3a3609453f9f76730f68829f39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 922f6e3a3609453f9f76730f68829f39, State: Initialized, Role: FOLLOWER
I20260806 22:47:28.802724 17844 consensus_queue.cc:260] T 00000000000000000000000000000000 P 922f6e3a3609453f9f76730f68829f39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "922f6e3a3609453f9f76730f68829f39" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 44911 } }
I20260806 22:47:28.802834 17844 raft_consensus.cc:388] T 00000000000000000000000000000000 P 922f6e3a3609453f9f76730f68829f39 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:47:28.802881 17844 raft_consensus.cc:482] T 00000000000000000000000000000000 P 922f6e3a3609453f9f76730f68829f39 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:47:28.802934 17844 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 922f6e3a3609453f9f76730f68829f39 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:47:28.803627 17844 raft_consensus.cc:504] T 00000000000000000000000000000000 P 922f6e3a3609453f9f76730f68829f39 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "922f6e3a3609453f9f76730f68829f39" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 44911 } }
I20260806 22:47:28.803756 17844 leader_election.cc:302] T 00000000000000000000000000000000 P 922f6e3a3609453f9f76730f68829f39 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 922f6e3a3609453f9f76730f68829f39; no voters:
I20260806 22:47:28.804003 17844 leader_election.cc:288] T 00000000000000000000000000000000 P 922f6e3a3609453f9f76730f68829f39 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:47:28.804035 17851 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 922f6e3a3609453f9f76730f68829f39 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:47:28.804176 17851 raft_consensus.cc:686] T 00000000000000000000000000000000 P 922f6e3a3609453f9f76730f68829f39 [term 1 LEADER]: Becoming Leader. State: Replica: 922f6e3a3609453f9f76730f68829f39, State: Running, Role: LEADER
I20260806 22:47:28.804301 17844 sys_catalog.cc:564] T 00000000000000000000000000000000 P 922f6e3a3609453f9f76730f68829f39 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:47:28.804370 17851 consensus_queue.cc:237] T 00000000000000000000000000000000 P 922f6e3a3609453f9f76730f68829f39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "922f6e3a3609453f9f76730f68829f39" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 44911 } }
I20260806 22:47:28.804824 17851 sys_catalog.cc:455] T 00000000000000000000000000000000 P 922f6e3a3609453f9f76730f68829f39 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 922f6e3a3609453f9f76730f68829f39. Latest consensus state: current_term: 1 leader_uuid: "922f6e3a3609453f9f76730f68829f39" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "922f6e3a3609453f9f76730f68829f39" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 44911 } } }
I20260806 22:47:28.804952 17853 sys_catalog.cc:455] T 00000000000000000000000000000000 P 922f6e3a3609453f9f76730f68829f39 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "922f6e3a3609453f9f76730f68829f39" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "922f6e3a3609453f9f76730f68829f39" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 44911 } } }
I20260806 22:47:28.805051 17853 sys_catalog.cc:458] T 00000000000000000000000000000000 P 922f6e3a3609453f9f76730f68829f39 [sys.catalog]: This master's current role is: LEADER
I20260806 22:47:28.805003 17851 sys_catalog.cc:458] T 00000000000000000000000000000000 P 922f6e3a3609453f9f76730f68829f39 [sys.catalog]: This master's current role is: LEADER
Aug 06 22:47:28 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056448, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
W20260806 22:47:28.806540 17869 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 922f6e3a3609453f9f76730f68829f39: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:47:28.806722 17869 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:47:28.806851 17858 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:47:28.807052 17858 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:47:28.808750 17858 catalog_manager.cc:1347] Generated new cluster ID: 4dcf6afc5fc847659b35fafafa202061
I20260806 22:47:28.808806 17858 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:47:28.810456 17847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:28.798979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52824 (local address 127.6.40.126:44911)
0806 22:47:28.799200 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:28.799211 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:28.799263 (+ 52us) server_negotiation.cc:407] Connection header received
0806 22:47:28.799764 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:28.799771 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:28.799820 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:28.799966 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:28.800685 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:28.801624 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:28.802425 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:28.802974 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:28.803082 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:28.806312 (+ 3230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:28.806317 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:28.806317 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:28.806332 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:28.808615 (+ 2283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:28.809128 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:28.809132 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:28.809132 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:28.809242 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:28.809495 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:28.809496 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:28.809497 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:28.809759 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:28.810000 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:28.810088 (+ 88us) server_negotiation.cc:299] Negotiation successful
0806 22:47:28.810244 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":103,"threads_started":1}
I20260806 22:47:28.817730 17858 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:47:28.818470 17858 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:47:28.824878 17858 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 922f6e3a3609453f9f76730f68829f39: Generated new TSK 0
I20260806 22:47:28.825054 17858 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.40.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.40.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.40.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.65.keytab.
Entry for principal kudu/127.6.40.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.65.keytab.
I20260806 22:47:28.864081 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.65:0
--local_ip_for_outbound_sockets=127.6.40.65
--webserver_interface=127.6.40.65
--webserver_port=0
--tserver_master_addrs=127.6.40.126:44911
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.65.keytab
--principal=kudu/127.6.40.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:45855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:47:28.969220 17875 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:47:28.969437 17875 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:47:28.969462 17875 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:47:28.970737 17875 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:47:28.970779 17875 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:47:28.970826 17875 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.65
I20260806 22:47:28.972294 17875 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:45855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.65.keytab
--principal=kudu/127.6.40.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.40.65
--webserver_port=0
--tserver_master_addrs=127.6.40.126:44911
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17875
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.65
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:47:28.972564 17875 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:47:28.972816 17875 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:47:28.978016 17881 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:28.978096 17886 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:28.978055 17884 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:28.978222 17875 server_base.cc:1034] running on GCE node
I20260806 22:47:28.978575 17875 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:47:28.978854 17875 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:47:28.980029 17875 hybrid_clock.cc:648] HybridClock initialized: now 1786056448979989 us; error 49 us; skew 500 ppm
Aug 06 22:47:28 dist-test-slave-zh2d krb5kdc[17732](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056448, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:47:28.982751 17875 init.cc:377] Logged in from keytab as kudu/127.6.40.65@KRBTEST.COM (short username kudu)
I20260806 22:47:28.983242 17875 webserver.cc:460] Webserver started at http://127.6.40.65:39521/ using document root <none> and password file <none>
I20260806 22:47:28.983480 17875 fs_manager.cc:359] Metadata directory not provided
I20260806 22:47:28.983525 17875 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:47:28.983671 17875 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:47:28.984829 17875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/ts-0/data/instance:
uuid: "03f092f9362c480f90418068817284ad"
format_stamp: "Formatted at 2026-08-06 22:47:28 on dist-test-slave-zh2d"
server_key: "e7e6c67c586117087b9826a8701f2351"
server_key_iv: "08a9d20addf8801b24162c679b25207b"
server_key_version: "encryptionkey@0"
I20260806 22:47:28.985275 17875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/ts-0/wal/instance:
uuid: "03f092f9362c480f90418068817284ad"
format_stamp: "Formatted at 2026-08-06 22:47:28 on dist-test-slave-zh2d"
server_key: "e7e6c67c586117087b9826a8701f2351"
server_key_iv: "08a9d20addf8801b24162c679b25207b"
server_key_version: "encryptionkey@0"
I20260806 22:47:28.986646 17875 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:47:28.987484 17896 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:28.987679 17875 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:47:28.987808 17875 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/ts-0/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/ts-0/wal
uuid: "03f092f9362c480f90418068817284ad"
format_stamp: "Formatted at 2026-08-06 22:47:28 on dist-test-slave-zh2d"
server_key: "e7e6c67c586117087b9826a8701f2351"
server_key_iv: "08a9d20addf8801b24162c679b25207b"
server_key_version: "encryptionkey@0"
I20260806 22:47:28.987895 17875 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:47:29.003402 17875 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:47:29.005914 17875 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:47:29.006094 17875 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:47:29.006433 17875 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:47:29.006803 17875 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:47:29.006865 17875 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:29.006914 17875 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:47:29.006946 17875 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:29.013909 17875 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.65:46303
I20260806 22:47:29.013975 18047 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.65:46303 every 8 connection(s)
I20260806 22:47:29.014317 17875 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
Aug 06 22:47:29 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056448, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.65@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
I20260806 22:47:29.020951 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 17875
I20260806 22:47:29.021095 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/ts-0/wal/instance
I20260806 22:47:29.021339 6305 external_mini_cluster.cc:1428] Setting key cdccec56724b3d2251b20c825a35097b
I20260806 22:47:29.024066 17847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:29.015014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.65:38477 (local address 127.6.40.126:44911)
0806 22:47:29.015060 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:29.015063 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:29.015350 (+ 287us) server_negotiation.cc:407] Connection header received
0806 22:47:29.015824 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:29.015825 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:29.015837 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:29.015906 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:29.016921 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:29.017450 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:29.018453 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:29.018669 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:29.018728 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:29.021033 (+ 2305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:29.021034 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:29.021035 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:29.021043 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:29.022616 (+ 1573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:29.023056 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:29.023057 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:29.023058 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:29.023124 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:29.023379 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:29.023380 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:29.023381 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:29.023502 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:29.023581 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:29.023859 (+ 278us) server_negotiation.cc:299] Negotiation successful
0806 22:47:29.023970 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 22:47:29.024237 18050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:29.014976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:44911 (local address 127.6.40.65:38477)
0806 22:47:29.015172 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:29.015216 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:29.015733 (+ 517us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:29.015978 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:29.015982 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:29.016579 (+ 597us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:29.016776 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:29.016781 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:29.017542 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:29.017543 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:29.018342 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:29.018344 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:29.018750 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:29.018750 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:29.018853 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:29.019355 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:29.019363 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:29.020923 (+ 1560us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:47:29.022734 (+ 1811us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:29.022742 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:29.022745 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:29.022976 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:29.023194 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:29.023195 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:29.023195 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:29.023306 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:29.023572 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:47:29.023579 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:47:29.023715 (+ 136us) client_negotiation.cc:769] Sending connection context
0806 22:47:29.023825 (+ 110us) client_negotiation.cc:240] Negotiation successful
0806 22:47:29.023989 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":114,"threads_started":1}
I20260806 22:47:29.024763 18048 heartbeater.cc:347] Connected to a master server at 127.6.40.126:44911
I20260806 22:47:29.024894 18048 heartbeater.cc:464] Registering TS with master...
I20260806 22:47:29.025117 18048 heartbeater.cc:511] Master 127.6.40.126:44911 requested a full tablet report, sending...
I20260806 22:47:29.025666 17792 ts_manager.cc:194] Registered new tserver with Master: 03f092f9362c480f90418068817284ad (127.6.40.65:46303)
I20260806 22:47:29.026309 17792 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.40.65@KRBTEST.COM'} at 127.6.40.65:38477
WARNING: no policy specified for kudu/127.6.40.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.40.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.40.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.66.keytab.
Entry for principal kudu/127.6.40.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.66.keytab.
I20260806 22:47:29.048418 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.66:0
--local_ip_for_outbound_sockets=127.6.40.66
--webserver_interface=127.6.40.66
--webserver_port=0
--tserver_master_addrs=127.6.40.126:44911
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.66.keytab
--principal=kudu/127.6.40.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:45855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:47:29.131888 18053 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:47:29.132099 18053 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:47:29.132126 18053 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:47:29.133339 18053 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:47:29.133383 18053 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:47:29.133409 18053 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.66
I20260806 22:47:29.134841 18053 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:45855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.66.keytab
--principal=kudu/127.6.40.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.40.66
--webserver_port=0
--tserver_master_addrs=127.6.40.126:44911
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18053
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.66
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:47:29.135090 18053 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:47:29.135324 18053 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:47:29.140818 18060 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:29.140939 18066 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:29.140952 18061 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:29.141011 18053 server_base.cc:1034] running on GCE node
I20260806 22:47:29.141325 18053 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:47:29.141525 18053 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:47:29.142663 18053 hybrid_clock.cc:648] HybridClock initialized: now 1786056449142647 us; error 41 us; skew 500 ppm
Aug 06 22:47:29 dist-test-slave-zh2d krb5kdc[17732](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056449, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:47:29.145232 18053 init.cc:377] Logged in from keytab as kudu/127.6.40.66@KRBTEST.COM (short username kudu)
I20260806 22:47:29.145722 18053 webserver.cc:460] Webserver started at http://127.6.40.66:44815/ using document root <none> and password file <none>
I20260806 22:47:29.145951 18053 fs_manager.cc:359] Metadata directory not provided
I20260806 22:47:29.145994 18053 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:47:29.146149 18053 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:47:29.147184 18053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/ts-1/data/instance:
uuid: "a90257621a934f52a3916fbb84ca6950"
format_stamp: "Formatted at 2026-08-06 22:47:29 on dist-test-slave-zh2d"
server_key: "12971e39aa088e70e30d369ac40e0057"
server_key_iv: "913d90113287ff7065fe00d12bea1336"
server_key_version: "encryptionkey@0"
I20260806 22:47:29.147555 18053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/ts-1/wal/instance:
uuid: "a90257621a934f52a3916fbb84ca6950"
format_stamp: "Formatted at 2026-08-06 22:47:29 on dist-test-slave-zh2d"
server_key: "12971e39aa088e70e30d369ac40e0057"
server_key_iv: "913d90113287ff7065fe00d12bea1336"
server_key_version: "encryptionkey@0"
I20260806 22:47:29.148931 18053 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:47:29.149756 18075 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:29.149984 18053 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:47:29.150086 18053 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/ts-1/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/ts-1/wal
uuid: "a90257621a934f52a3916fbb84ca6950"
format_stamp: "Formatted at 2026-08-06 22:47:29 on dist-test-slave-zh2d"
server_key: "12971e39aa088e70e30d369ac40e0057"
server_key_iv: "913d90113287ff7065fe00d12bea1336"
server_key_version: "encryptionkey@0"
I20260806 22:47:29.150163 18053 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:47:29.162143 18053 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:47:29.164309 18053 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:47:29.164479 18053 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:47:29.164705 18053 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:47:29.165043 18053 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:47:29.165078 18053 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:29.165128 18053 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:47:29.165153 18053 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:29.171031 18053 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.66:38465
I20260806 22:47:29.171130 18234 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.66:38465 every 8 connection(s)
I20260806 22:47:29.171437 18053 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
I20260806 22:47:29.173444 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 18053
I20260806 22:47:29.173563 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/ts-1/wal/instance
I20260806 22:47:29.173760 6305 external_mini_cluster.cc:1428] Setting key 38bd34138022a45ac9271cb0ee242a7d
Aug 06 22:47:29 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056449, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.66@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
I20260806 22:47:29.182464 17847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:29.172176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.66:36665 (local address 127.6.40.126:44911)
0806 22:47:29.172249 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:29.172252 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:29.172451 (+ 199us) server_negotiation.cc:407] Connection header received
0806 22:47:29.173003 (+ 552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:29.173004 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:29.173014 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:29.173080 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:29.174154 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:29.174816 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:29.176351 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:29.176684 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:29.176769 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:29.179380 (+ 2611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:29.179381 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:29.179382 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:29.179388 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:29.180952 (+ 1564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:29.181464 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:29.181465 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:29.181465 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:29.181529 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:29.181827 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:29.181827 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:29.181827 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:29.181928 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:29.181979 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:29.182326 (+ 347us) server_negotiation.cc:299] Negotiation successful
0806 22:47:29.182397 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 22:47:29.182753 18237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:29.172142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:44911 (local address 127.6.40.66:36665)
0806 22:47:29.172332 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:29.172368 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:29.172906 (+ 538us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:29.173137 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:29.173140 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:29.173789 (+ 649us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:29.174041 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:29.174048 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:29.174914 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:29.174916 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:29.176235 (+ 1319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:29.176239 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:29.176782 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:29.176783 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:29.176977 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:29.177501 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:29.177509 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:29.179280 (+ 1771us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:47:29.181034 (+ 1754us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:29.181041 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:29.181043 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:29.181380 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:29.181594 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:29.181595 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:29.181596 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:29.181753 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:29.181997 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:47:29.182005 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:47:29.182181 (+ 176us) client_negotiation.cc:769] Sending connection context
0806 22:47:29.182309 (+ 128us) client_negotiation.cc:240] Negotiation successful
0806 22:47:29.182499 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":104,"threads_started":1}
I20260806 22:47:29.183303 18235 heartbeater.cc:347] Connected to a master server at 127.6.40.126:44911
I20260806 22:47:29.183400 18235 heartbeater.cc:464] Registering TS with master...
I20260806 22:47:29.183612 18235 heartbeater.cc:511] Master 127.6.40.126:44911 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.6.40.67@KRBTEST.COM; defaulting to no policy
I20260806 22:47:29.184026 17792 ts_manager.cc:194] Registered new tserver with Master: a90257621a934f52a3916fbb84ca6950 (127.6.40.66:38465)
I20260806 22:47:29.184406 17792 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.40.66@KRBTEST.COM'} at 127.6.40.66:36665
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.40.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.40.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.67.keytab.
Entry for principal kudu/127.6.40.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.67.keytab.
I20260806 22:47:29.205031 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.67:0
--local_ip_for_outbound_sockets=127.6.40.67
--webserver_interface=127.6.40.67
--webserver_port=0
--tserver_master_addrs=127.6.40.126:44911
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.67.keytab
--principal=kudu/127.6.40.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:45855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:47:29.290350 18244 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:47:29.290639 18244 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:47:29.290680 18244 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:47:29.292098 18244 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:47:29.292156 18244 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:47:29.292202 18244 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.67
I20260806 22:47:29.293737 18244 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:45855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.67.keytab
--principal=kudu/127.6.40.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.40.67
--webserver_port=0
--tserver_master_addrs=127.6.40.126:44911
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18244
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.67
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:47:29.294031 18244 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:47:29.294272 18244 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:47:29.299623 18254 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:29.299736 18251 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:29.299660 18252 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:29.299999 18244 server_base.cc:1034] running on GCE node
I20260806 22:47:29.300319 18244 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:47:29.300595 18244 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:47:29.301777 18244 hybrid_clock.cc:648] HybridClock initialized: now 1786056449301751 us; error 41 us; skew 500 ppm
Aug 06 22:47:29 dist-test-slave-zh2d krb5kdc[17732](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056449, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:47:29.304641 18244 init.cc:377] Logged in from keytab as kudu/127.6.40.67@KRBTEST.COM (short username kudu)
I20260806 22:47:29.305200 18244 webserver.cc:460] Webserver started at http://127.6.40.67:39275/ using document root <none> and password file <none>
I20260806 22:47:29.305480 18244 fs_manager.cc:359] Metadata directory not provided
I20260806 22:47:29.305548 18244 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:47:29.305653 18244 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:47:29.306798 18244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/ts-2/data/instance:
uuid: "5c7f5c0d6fdc448c936edb5ed4825635"
format_stamp: "Formatted at 2026-08-06 22:47:29 on dist-test-slave-zh2d"
server_key: "ad3a10f01f7ba7d34662e91523bc9ea0"
server_key_iv: "62d1ace97b29b33ac5d1d0e3efe95152"
server_key_version: "encryptionkey@0"
I20260806 22:47:29.307122 18244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/ts-2/wal/instance:
uuid: "5c7f5c0d6fdc448c936edb5ed4825635"
format_stamp: "Formatted at 2026-08-06 22:47:29 on dist-test-slave-zh2d"
server_key: "ad3a10f01f7ba7d34662e91523bc9ea0"
server_key_iv: "62d1ace97b29b33ac5d1d0e3efe95152"
server_key_version: "encryptionkey@0"
I20260806 22:47:29.308524 18244 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:47:29.309387 18261 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:29.309594 18244 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 22:47:29.309682 18244 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/ts-2/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/ts-2/wal
uuid: "5c7f5c0d6fdc448c936edb5ed4825635"
format_stamp: "Formatted at 2026-08-06 22:47:29 on dist-test-slave-zh2d"
server_key: "ad3a10f01f7ba7d34662e91523bc9ea0"
server_key_iv: "62d1ace97b29b33ac5d1d0e3efe95152"
server_key_version: "encryptionkey@0"
I20260806 22:47:29.309739 18244 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:47:29.318760 18244 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:47:29.320616 18244 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:47:29.320729 18244 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:47:29.320968 18244 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:47:29.321334 18244 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:47:29.321370 18244 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:29.321393 18244 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:47:29.321429 18244 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:29.327816 18244 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.67:40059
I20260806 22:47:29.327921 18410 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.67:40059 every 8 connection(s)
I20260806 22:47:29.328243 18244 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
I20260806 22:47:29.330639 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 18244
I20260806 22:47:29.330785 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/ts-2/wal/instance
I20260806 22:47:29.331056 6305 external_mini_cluster.cc:1428] Setting key 87103ada35518df96c48c33f0996b48a
Aug 06 22:47:29 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056449, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.67@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
I20260806 22:47:29.338341 17847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:29.329043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.67:40051 (local address 127.6.40.126:44911)
0806 22:47:29.329105 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:29.329107 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:29.329362 (+ 255us) server_negotiation.cc:407] Connection header received
0806 22:47:29.329873 (+ 511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:29.329874 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:29.329889 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:29.329945 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:29.330958 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:29.331774 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:29.332981 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:29.333183 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:29.333247 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:29.335442 (+ 2195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:29.335444 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:29.335444 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:29.335451 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:29.337068 (+ 1617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:29.337454 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:29.337455 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:29.337456 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:29.337517 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:29.337747 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:29.337747 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:29.337748 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:29.337866 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:29.337921 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:29.338199 (+ 278us) server_negotiation.cc:299] Negotiation successful
0806 22:47:29.338272 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 22:47:29.338491 18413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:29.329045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:44911 (local address 127.6.40.67:40051)
0806 22:47:29.329208 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:29.329245 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:29.329772 (+ 527us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:29.329999 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:29.330003 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:29.330603 (+ 600us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:29.330857 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:29.330863 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:29.331877 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:29.331878 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:29.332832 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:29.332874 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:29.333266 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:29.333267 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:29.333383 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:29.333879 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:29.333888 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:29.335336 (+ 1448us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:47:29.337141 (+ 1805us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:29.337148 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:29.337149 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:29.337367 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:29.337588 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:29.337588 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:29.337589 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:29.337687 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:29.337925 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:47:29.337929 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:47:29.338053 (+ 124us) client_negotiation.cc:769] Sending connection context
0806 22:47:29.338139 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 22:47:29.338265 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":85,"threads_started":1}
I20260806 22:47:29.339047 18411 heartbeater.cc:347] Connected to a master server at 127.6.40.126:44911
I20260806 22:47:29.339146 18411 heartbeater.cc:464] Registering TS with master...
I20260806 22:47:29.339306 18411 heartbeater.cc:511] Master 127.6.40.126:44911 requested a full tablet report, sending...
I20260806 22:47:29.339686 17792 ts_manager.cc:194] Registered new tserver with Master: 5c7f5c0d6fdc448c936edb5ed4825635 (127.6.40.67:40059)
I20260806 22:47:29.340080 17792 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.40.67@KRBTEST.COM'} at 127.6.40.67:40051
I20260806 22:47:29.342933 6305 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.6.40.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.40.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.40.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/krb5kdc/oryx_127.6.40.65.keytab.
Entry for principal oryx/127.6.40.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/krb5kdc/oryx_127.6.40.65.keytab.
WARNING: no policy specified for oryx/127.6.40.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.40.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.40.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/krb5kdc/oryx_127.6.40.66.keytab.
Entry for principal oryx/127.6.40.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/krb5kdc/oryx_127.6.40.66.keytab.
WARNING: no policy specified for oryx/127.6.40.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.40.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.40.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/krb5kdc/oryx_127.6.40.67.keytab.
Entry for principal oryx/127.6.40.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/krb5kdc/oryx_127.6.40.67.keytab.
I20260806 22:47:29.428447 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 17875
I20260806 22:47:29.446666 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 18053
I20260806 22:47:29.456733 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 18244
I20260806 22:47:29.468600 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 17749
I20260806 22:47:29.476047 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.126:44911
--webserver_interface=127.6.40.126
--webserver_port=46729
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab
--principal=kudu/127.6.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:45855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.40.126:44911
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:47:29.561530 18423 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:47:29.561741 18423 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:47:29.561764 18423 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:47:29.562865 18423 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:47:29.562901 18423 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:47:29.562913 18423 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:47:29.562927 18423 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:47:29.562938 18423 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:47:29.564386 18423 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:45855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.40.126:44911
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab
--principal=kudu/127.6.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.126:44911
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.40.126
--webserver_port=46729
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18423
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:47:29.564602 18423 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:47:29.564801 18423 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:47:29.569854 18431 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:29.569877 18423 server_base.cc:1034] running on GCE node
W20260806 22:47:29.569856 18429 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:29.569873 18428 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:29.570393 18423 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:47:29.570693 18423 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:47:29.571880 18423 hybrid_clock.cc:648] HybridClock initialized: now 1786056449571862 us; error 28 us; skew 500 ppm
Aug 06 22:47:29 dist-test-slave-zh2d krb5kdc[17732](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056449, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:47:29.574497 18423 init.cc:377] Logged in from keytab as kudu/127.6.40.126@KRBTEST.COM (short username kudu)
I20260806 22:47:29.574987 18423 webserver.cc:460] Webserver started at http://127.6.40.126:46729/ using document root <none> and password file <none>
I20260806 22:47:29.575243 18423 fs_manager.cc:359] Metadata directory not provided
I20260806 22:47:29.575337 18423 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:47:29.576742 18423 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:47:29.577466 18441 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:29.578905 18423 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.000s
I20260806 22:47:29.579015 18423 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/master-0/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/master-0/wal
uuid: "922f6e3a3609453f9f76730f68829f39"
format_stamp: "Formatted at 2026-08-06 22:47:28 on dist-test-slave-zh2d"
server_key: "12bcc3aa6829694d9fc36e61ec2c0164"
server_key_iv: "e805aff0d6521c77f34f8cbb52b7dbcd"
server_key_version: "encryptionkey@0"
I20260806 22:47:29.579403 18423 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:47:29.587750 18423 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:47:29.589763 18423 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:47:29.589936 18423 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:47:29.594480 18423 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.126:44911
I20260806 22:47:29.594532 18513 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.126:44911 every 8 connection(s)
I20260806 22:47:29.594918 18423 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
I20260806 22:47:29.596463 18515 sys_catalog.cc:263] Verifying existing consensus state
I20260806 22:47:29.597105 18515 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 922f6e3a3609453f9f76730f68829f39: Bootstrap starting.
I20260806 22:47:29.601626 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 18423
I20260806 22:47:29.601976 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.65:46303
--local_ip_for_outbound_sockets=127.6.40.65
--tserver_master_addrs=127.6.40.126:44911
--webserver_port=39521
--webserver_interface=127.6.40.65
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.65.keytab
--principal=kudu/127.6.40.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:45855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.6.40.65
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/krb5kdc/oryx_127.6.40.65.keytab with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:47:29.606920 18515 log.cc:824] T 00000000000000000000000000000000 P 922f6e3a3609453f9f76730f68829f39: Log is configured to *not* fsync() on all Append() calls
I20260806 22:47:29.608700 18515 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 922f6e3a3609453f9f76730f68829f39: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 22:47:29.609089 18515 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 922f6e3a3609453f9f76730f68829f39: Bootstrap complete.
I20260806 22:47:29.611477 18515 raft_consensus.cc:348] T 00000000000000000000000000000000 P 922f6e3a3609453f9f76730f68829f39 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "922f6e3a3609453f9f76730f68829f39" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 44911 } }
I20260806 22:47:29.611729 18515 raft_consensus.cc:729] T 00000000000000000000000000000000 P 922f6e3a3609453f9f76730f68829f39 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 922f6e3a3609453f9f76730f68829f39, State: Initialized, Role: FOLLOWER
I20260806 22:47:29.611943 18515 consensus_queue.cc:260] T 00000000000000000000000000000000 P 922f6e3a3609453f9f76730f68829f39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "922f6e3a3609453f9f76730f68829f39" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 44911 } }
I20260806 22:47:29.612054 18515 raft_consensus.cc:388] T 00000000000000000000000000000000 P 922f6e3a3609453f9f76730f68829f39 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:47:29.612111 18515 raft_consensus.cc:482] T 00000000000000000000000000000000 P 922f6e3a3609453f9f76730f68829f39 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:47:29.612161 18515 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 922f6e3a3609453f9f76730f68829f39 [term 1 FOLLOWER]: Advancing to term 2
I20260806 22:47:29.613267 18515 raft_consensus.cc:504] T 00000000000000000000000000000000 P 922f6e3a3609453f9f76730f68829f39 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "922f6e3a3609453f9f76730f68829f39" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 44911 } }
I20260806 22:47:29.613392 18515 leader_election.cc:302] T 00000000000000000000000000000000 P 922f6e3a3609453f9f76730f68829f39 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 922f6e3a3609453f9f76730f68829f39; no voters:
I20260806 22:47:29.613600 18515 leader_election.cc:288] T 00000000000000000000000000000000 P 922f6e3a3609453f9f76730f68829f39 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 22:47:29.613638 18519 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 922f6e3a3609453f9f76730f68829f39 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 22:47:29.613768 18519 raft_consensus.cc:686] T 00000000000000000000000000000000 P 922f6e3a3609453f9f76730f68829f39 [term 2 LEADER]: Becoming Leader. State: Replica: 922f6e3a3609453f9f76730f68829f39, State: Running, Role: LEADER
I20260806 22:47:29.613905 18515 sys_catalog.cc:564] T 00000000000000000000000000000000 P 922f6e3a3609453f9f76730f68829f39 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:47:29.613930 18519 consensus_queue.cc:237] T 00000000000000000000000000000000 P 922f6e3a3609453f9f76730f68829f39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "922f6e3a3609453f9f76730f68829f39" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 44911 } }
I20260806 22:47:29.614326 18520 sys_catalog.cc:455] T 00000000000000000000000000000000 P 922f6e3a3609453f9f76730f68829f39 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "922f6e3a3609453f9f76730f68829f39" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "922f6e3a3609453f9f76730f68829f39" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 44911 } } }
I20260806 22:47:29.614413 18520 sys_catalog.cc:458] T 00000000000000000000000000000000 P 922f6e3a3609453f9f76730f68829f39 [sys.catalog]: This master's current role is: LEADER
I20260806 22:47:29.614660 18522 sys_catalog.cc:455] T 00000000000000000000000000000000 P 922f6e3a3609453f9f76730f68829f39 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 922f6e3a3609453f9f76730f68829f39. Latest consensus state: current_term: 2 leader_uuid: "922f6e3a3609453f9f76730f68829f39" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "922f6e3a3609453f9f76730f68829f39" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 44911 } } }
I20260806 22:47:29.614744 18522 sys_catalog.cc:458] T 00000000000000000000000000000000 P 922f6e3a3609453f9f76730f68829f39 [sys.catalog]: This master's current role is: LEADER
I20260806 22:47:29.615976 18541 catalog_manager.cc:1259] Loaded cluster ID: 4dcf6afc5fc847659b35fafafa202061
I20260806 22:47:29.616056 18541 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 922f6e3a3609453f9f76730f68829f39: loading cluster ID for follower catalog manager: success
I20260806 22:47:29.617267 18541 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 922f6e3a3609453f9f76730f68829f39: acquiring CA information for follower catalog manager: success
I20260806 22:47:29.617643 18541 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 922f6e3a3609453f9f76730f68829f39: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 22:47:29.617719 18528 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:47:29.617878 18528 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:47:29.617946 18528 catalog_manager.cc:1259] Loaded cluster ID: 4dcf6afc5fc847659b35fafafa202061
I20260806 22:47:29.617970 18528 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:47:29.618139 18528 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:47:29.618208 18528 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 922f6e3a3609453f9f76730f68829f39: Loaded TSK: 0
I20260806 22:47:29.618322 18528 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 22:47:29.690586 18517 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:47:29.690795 18517 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:47:29.690819 18517 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:47:29.692026 18517 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:47:29.692075 18517 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:47:29.692102 18517 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.65
I20260806 22:47:29.693812 18517 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:45855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/krb5kdc/oryx_127.6.40.65.keytab
--principal=oryx/127.6.40.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.65:46303
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.40.65
--webserver_port=39521
--tserver_master_addrs=127.6.40.126:44911
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18517
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.65
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:47:29.694092 18517 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:47:29.694346 18517 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:47:29.700958 18553 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:29.700978 18551 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:29.701134 18549 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:29.701053 18517 server_base.cc:1034] running on GCE node
I20260806 22:47:29.701545 18517 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:47:29.701814 18517 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:47:29.703001 18517 hybrid_clock.cc:648] HybridClock initialized: now 1786056449702962 us; error 55 us; skew 500 ppm
Aug 06 22:47:29 dist-test-slave-zh2d krb5kdc[17732](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056449, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.40.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:47:29.705767 18517 init.cc:377] Logged in from keytab as oryx/127.6.40.65@KRBTEST.COM (short username oryx)
I20260806 22:47:29.706311 18517 webserver.cc:460] Webserver started at http://127.6.40.65:39521/ using document root <none> and password file <none>
I20260806 22:47:29.706614 18517 fs_manager.cc:359] Metadata directory not provided
I20260806 22:47:29.706678 18517 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:47:29.708202 18517 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:47:29.708881 18560 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:29.709043 18517 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:47:29.709158 18517 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/ts-0/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/ts-0/wal
uuid: "03f092f9362c480f90418068817284ad"
format_stamp: "Formatted at 2026-08-06 22:47:28 on dist-test-slave-zh2d"
server_key: "e7e6c67c586117087b9826a8701f2351"
server_key_iv: "08a9d20addf8801b24162c679b25207b"
server_key_version: "encryptionkey@0"
I20260806 22:47:29.709537 18517 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:47:29.727350 18517 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:47:29.729948 18517 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:47:29.730137 18517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:47:29.730540 18517 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:47:29.730931 18517 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:47:29.730985 18517 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:29.731029 18517 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:47:29.731055 18517 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:29.738754 18517 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.65:46303
I20260806 22:47:29.738844 18710 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.65:46303 every 8 connection(s)
I20260806 22:47:29.739192 18517 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
Aug 06 22:47:29 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.65@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:47:29 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.65@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:47:29.748272 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 18517
I20260806 22:47:29.748613 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.66:38465
--local_ip_for_outbound_sockets=127.6.40.66
--tserver_master_addrs=127.6.40.126:44911
--webserver_port=44815
--webserver_interface=127.6.40.66
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.66.keytab
--principal=kudu/127.6.40.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:45855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.6.40.66
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/krb5kdc/oryx_127.6.40.66.keytab with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:47:29.749380 18715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:29.739932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:44911 (local address 127.6.40.65:59023)
0806 22:47:29.740149 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:29.740199 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:29.740899 (+ 700us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:29.741388 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:29.741393 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:29.742173 (+ 780us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:29.742533 (+ 360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:29.742541 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:29.743696 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:29.743697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:29.744990 (+ 1293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:29.744994 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:29.745613 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:29.745614 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:29.745758 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:29.746370 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:29.746380 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:29.749136 (+ 2756us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.40.126:44911: Server oryx/127.6.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":98,"threads_started":1}
W20260806 22:47:29.749439 18718 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:47:29.740057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.65:59023 (local address 127.6.40.126:44911)
0806 22:47:29.740257 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:29.740264 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:29.740292 (+ 28us) server_negotiation.cc:407] Connection header received
0806 22:47:29.741124 (+ 832us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:29.741130 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:29.741177 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:29.741295 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:29.742687 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:29.743586 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:29.745079 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:29.745517 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:29.745607 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:29.749272 (+ 3665us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.40.65:59023: BlockingRecv error: recv got EOF from 127.6.40.65:59023 (error 108)
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":79,"threads_started":1}
W20260806 22:47:29.749773 18711 heartbeater.cc:644] Failed to heartbeat to 127.6.40.126:44911 (0 consecutive failures): Not authorized: Failed to ping master at 127.6.40.126:44911: Client connection negotiation failed: client connection to 127.6.40.126:44911: Server oryx/127.6.40.126@KRBTEST.COM not found in Kerberos database
Aug 06 22:47:29 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.65@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:47:29 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.65@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:47:29.755249 18715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:29.750073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:44911 (local address 127.6.40.65:55045)
0806 22:47:29.750220 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:29.750232 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:29.750327 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:29.750547 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:29.750548 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:29.750723 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:29.750784 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:29.750786 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:29.751605 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:29.751606 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:29.752166 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:29.752168 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:29.752568 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:29.752569 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:29.752676 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:29.753209 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:29.753214 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:29.755153 (+ 1939us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.40.126:44911: Server oryx/127.6.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":77,"threads_started":1}
W20260806 22:47:29.755308 18718 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:47:29.750128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.65:55045 (local address 127.6.40.126:44911)
0806 22:47:29.750171 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:29.750174 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:29.750280 (+ 106us) server_negotiation.cc:407] Connection header received
0806 22:47:29.750422 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:29.750423 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:29.750439 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:29.750477 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:29.750883 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:29.751513 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:29.752278 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:29.752476 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:29.752568 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:29.755241 (+ 2673us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.40.65:55045: BlockingRecv error: recv got EOF from 127.6.40.65:55045 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Aug 06 22:47:29 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.65@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:47:29 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.65@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:47:29.760121 18715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:29.755717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:44911 (local address 127.6.40.65:52545)
0806 22:47:29.755768 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:29.755780 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:29.755845 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:29.755989 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:29.755990 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:29.756127 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:29.756155 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:29.756157 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:29.756808 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:29.756809 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:29.757349 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:29.757351 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:29.757756 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:29.757757 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:29.757813 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:29.758290 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:29.758304 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:29.760032 (+ 1728us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.40.126:44911: Server oryx/127.6.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":33}
W20260806 22:47:29.760182 18718 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:47:29.755807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.65:52545 (local address 127.6.40.126:44911)
0806 22:47:29.755873 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:29.755875 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:29.755887 (+ 12us) server_negotiation.cc:407] Connection header received
0806 22:47:29.755910 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:29.755911 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:29.755923 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:29.755985 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:29.756245 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:29.756706 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:29.757458 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:29.757661 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:29.757754 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:29.760114 (+ 2360us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.40.65:52545: BlockingRecv error: recv got EOF from 127.6.40.65:52545 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260806 22:47:29.760433 18711 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 22:47:29.835990 18720 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:47:29.836201 18720 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:47:29.836225 18720 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:47:29.837437 18720 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:47:29.837479 18720 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:47:29.837505 18720 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.66
I20260806 22:47:29.838938 18720 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:45855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/krb5kdc/oryx_127.6.40.66.keytab
--principal=oryx/127.6.40.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.66:38465
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.40.66
--webserver_port=44815
--tserver_master_addrs=127.6.40.126:44911
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18720
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.66
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:47:29.839206 18720 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:47:29.839445 18720 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:47:29.844770 18727 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:29.844807 18728 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:29.845048 18720 server_base.cc:1034] running on GCE node
W20260806 22:47:29.845235 18731 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:29.845463 18720 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:47:29.845745 18720 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:47:29.846946 18720 hybrid_clock.cc:648] HybridClock initialized: now 1786056449846931 us; error 35 us; skew 500 ppm
Aug 06 22:47:29 dist-test-slave-zh2d krb5kdc[17732](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056449, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.40.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:47:29.849857 18720 init.cc:377] Logged in from keytab as oryx/127.6.40.66@KRBTEST.COM (short username oryx)
I20260806 22:47:29.850351 18720 webserver.cc:460] Webserver started at http://127.6.40.66:44815/ using document root <none> and password file <none>
I20260806 22:47:29.850612 18720 fs_manager.cc:359] Metadata directory not provided
I20260806 22:47:29.850672 18720 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:47:29.852135 18720 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:47:29.853076 18738 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:29.853271 18720 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:47:29.853375 18720 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/ts-1/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/ts-1/wal
uuid: "a90257621a934f52a3916fbb84ca6950"
format_stamp: "Formatted at 2026-08-06 22:47:29 on dist-test-slave-zh2d"
server_key: "12971e39aa088e70e30d369ac40e0057"
server_key_iv: "913d90113287ff7065fe00d12bea1336"
server_key_version: "encryptionkey@0"
I20260806 22:47:29.853750 18720 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:47:29.865115 18720 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:47:29.867914 18720 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:47:29.868080 18720 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:47:29.868330 18720 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:47:29.868654 18720 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:47:29.868688 18720 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:29.868711 18720 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:47:29.868757 18720 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:29.874936 18720 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.66:38465
I20260806 22:47:29.875012 18891 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.66:38465 every 8 connection(s)
I20260806 22:47:29.875372 18720 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
Aug 06 22:47:29 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.66@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:47:29 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.66@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
W20260806 22:47:29.882819 18718 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:47:29.876150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.66:49845 (local address 127.6.40.126:44911)
0806 22:47:29.876218 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:29.876221 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:29.876525 (+ 304us) server_negotiation.cc:407] Connection header received
0806 22:47:29.877096 (+ 571us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:29.877098 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:29.877113 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:29.877163 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:29.878045 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:29.878532 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:29.879545 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:29.879727 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:29.879776 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:29.882761 (+ 2985us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.40.66:49845: BlockingRecv error: recv got EOF from 127.6.40.66:49845 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 22:47:29.882880 18895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:29.876145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:44911 (local address 127.6.40.66:49845)
0806 22:47:29.876366 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:29.876427 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:29.877005 (+ 578us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:29.877220 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:29.877223 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:29.877779 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:29.877966 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:29.877971 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:29.878617 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:29.878618 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:29.879470 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:29.879472 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:29.879814 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:29.879815 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:29.879930 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:29.880402 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:29.880410 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:29.882684 (+ 2274us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.40.126:44911: Server oryx/127.6.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":107,"threads_started":1}
W20260806 22:47:29.883101 18892 heartbeater.cc:644] Failed to heartbeat to 127.6.40.126:44911 (0 consecutive failures): Not authorized: Failed to ping master at 127.6.40.126:44911: Client connection negotiation failed: client connection to 127.6.40.126:44911: Server oryx/127.6.40.126@KRBTEST.COM not found in Kerberos database
I20260806 22:47:29.885421 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 18720
I20260806 22:47:29.885747 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.67:40059
--local_ip_for_outbound_sockets=127.6.40.67
--tserver_master_addrs=127.6.40.126:44911
--webserver_port=39275
--webserver_interface=127.6.40.67
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.67.keytab
--principal=kudu/127.6.40.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:45855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.6.40.67
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/krb5kdc/oryx_127.6.40.67.keytab with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 22:47:29 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.66@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:47:29 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.66@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:47:29.888090 18895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:29.883338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:44911 (local address 127.6.40.66:37791)
0806 22:47:29.883439 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:29.883453 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:29.883519 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:29.883620 (+ 101us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:29.883621 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:29.883756 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:29.883807 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:29.883808 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:29.884407 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:29.884408 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:29.885031 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:29.885033 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:29.885452 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:29.885453 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:29.885541 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:29.886017 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:29.886022 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:29.888045 (+ 2023us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.40.126:44911: Server oryx/127.6.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":79}
W20260806 22:47:29.888235 18718 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:47:29.883411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.66:37791 (local address 127.6.40.126:44911)
0806 22:47:29.883472 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:29.883473 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:29.883503 (+ 30us) server_negotiation.cc:407] Connection header received
0806 22:47:29.883563 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:29.883564 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:29.883571 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:29.883616 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:29.883868 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:29.884317 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:29.885107 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:29.885375 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:29.885448 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:29.888151 (+ 2703us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.40.66:37791: BlockingRecv error: recv got EOF from 127.6.40.66:37791 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 06 22:47:29 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.66@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:47:29 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.66@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:47:29.893324 18897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:29.888328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:44911 (local address 127.6.40.66:35201)
0806 22:47:29.888469 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:29.888481 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:29.888548 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:29.888694 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:29.888695 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:29.888828 (+ 133us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:29.888919 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:29.888921 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:29.889562 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:29.889563 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:29.890126 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:29.890128 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:29.890663 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:29.890663 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:29.890707 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:29.891204 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:29.891208 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:29.893224 (+ 2016us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.40.126:44911: Server oryx/127.6.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":83,"threads_started":1}
W20260806 22:47:29.893399 18718 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:47:29.888500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.66:35201 (local address 127.6.40.126:44911)
0806 22:47:29.888555 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:29.888557 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:29.888570 (+ 13us) server_negotiation.cc:407] Connection header received
0806 22:47:29.888597 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:29.888598 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:29.888609 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:29.888687 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:29.888993 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:29.889485 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:29.890213 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:29.890555 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:29.890638 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:29.893332 (+ 2694us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.40.66:35201: BlockingRecv error: recv got EOF from 127.6.40.66:35201 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
W20260806 22:47:29.893447 18892 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 22:47:29.974081 18896 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:47:29.974336 18896 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:47:29.974390 18896 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:47:29.975579 18896 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:47:29.975648 18896 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:47:29.975692 18896 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.67
I20260806 22:47:29.977221 18896 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:45855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/krb5kdc/oryx_127.6.40.67.keytab
--principal=oryx/127.6.40.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.67:40059
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.40.67
--webserver_port=39275
--tserver_master_addrs=127.6.40.126:44911
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18896
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.67
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:47:29.977512 18896 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:47:29.977751 18896 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:47:29.983130 18906 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:29.983139 18910 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:29.983377 18896 server_base.cc:1034] running on GCE node
W20260806 22:47:29.983494 18907 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:29.983784 18896 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:47:29.983992 18896 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:47:29.985137 18896 hybrid_clock.cc:648] HybridClock initialized: now 1786056449985124 us; error 32 us; skew 500 ppm
Aug 06 22:47:29 dist-test-slave-zh2d krb5kdc[17732](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056449, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.40.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:47:29.987721 18896 init.cc:377] Logged in from keytab as oryx/127.6.40.67@KRBTEST.COM (short username oryx)
I20260806 22:47:29.988183 18896 webserver.cc:460] Webserver started at http://127.6.40.67:39275/ using document root <none> and password file <none>
I20260806 22:47:29.988409 18896 fs_manager.cc:359] Metadata directory not provided
I20260806 22:47:29.988469 18896 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:47:29.989974 18896 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:47:29.990648 18918 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:29.990836 18896 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:47:29.990934 18896 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/ts-2/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/ts-2/wal
uuid: "5c7f5c0d6fdc448c936edb5ed4825635"
format_stamp: "Formatted at 2026-08-06 22:47:29 on dist-test-slave-zh2d"
server_key: "ad3a10f01f7ba7d34662e91523bc9ea0"
server_key_iv: "62d1ace97b29b33ac5d1d0e3efe95152"
server_key_version: "encryptionkey@0"
I20260806 22:47:29.991308 18896 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:47:30.010674 18896 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:47:30.012989 18896 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:47:30.013181 18896 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:47:30.013465 18896 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:47:30.013821 18896 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:47:30.013856 18896 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:30.013880 18896 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:47:30.013978 18896 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:30.020097 18896 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.67:40059
I20260806 22:47:30.020201 19075 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.67:40059 every 8 connection(s)
I20260806 22:47:30.020522 18896 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
I20260806 22:47:30.023653 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 18896
Aug 06 22:47:30 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.67@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:47:30 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.67@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
W20260806 22:47:30.029210 18718 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:47:30.021343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.67:43565 (local address 127.6.40.126:44911)
0806 22:47:30.021430 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:30.021440 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:30.021614 (+ 174us) server_negotiation.cc:407] Connection header received
0806 22:47:30.022180 (+ 566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:30.022182 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:30.022195 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:30.022247 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:30.023233 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:30.023947 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:30.025487 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:30.025692 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:30.025778 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:30.029152 (+ 3374us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.40.67:43565: BlockingRecv error: recv got EOF from 127.6.40.67:43565 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 22:47:30.029270 19078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:30.021310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:44911 (local address 127.6.40.67:43565)
0806 22:47:30.021470 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:30.021515 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:30.022066 (+ 551us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:30.022330 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:30.022334 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:30.022924 (+ 590us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:30.023123 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:30.023128 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:30.024049 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:30.024051 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:30.025391 (+ 1340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:30.025393 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:30.025791 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:30.025792 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:30.025943 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:30.026483 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:30.026491 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:30.029007 (+ 2516us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.40.126:44911: Server oryx/127.6.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":75,"threads_started":1}
W20260806 22:47:30.029496 19076 heartbeater.cc:644] Failed to heartbeat to 127.6.40.126:44911 (0 consecutive failures): Not authorized: Failed to ping master at 127.6.40.126:44911: Client connection negotiation failed: client connection to 127.6.40.126:44911: Server oryx/127.6.40.126@KRBTEST.COM not found in Kerberos database
I20260806 22:47:30.031491 19082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:30.024183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52832 (local address 127.6.40.126:44911)
0806 22:47:30.024305 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:30.024306 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:30.024315 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:47:30.024356 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:30.024357 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:30.024367 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:30.024427 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:30.025191 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:30.025650 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:30.026428 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:30.026621 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:30.026697 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:30.028075 (+ 1378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:30.028079 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:30.028080 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:30.028090 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:30.030094 (+ 2004us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:30.030461 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:30.030465 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:30.030465 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:30.030590 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:30.030779 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:30.030780 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:30.030780 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:30.031071 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:30.031182 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:30.031313 (+ 131us) server_negotiation.cc:299] Negotiation successful
0806 22:47:30.031398 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":53,"threads_started":1}
Aug 06 22:47:30 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.67@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:47:30 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.67@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:47:30.034397 19078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:30.029738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:44911 (local address 127.6.40.67:47407)
0806 22:47:30.029770 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:30.029780 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:30.029840 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:30.029973 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:30.029974 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:30.030113 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:30.030185 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:30.030187 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:30.031042 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:30.031043 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:30.031813 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:30.031818 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:30.032172 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:30.032173 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:30.032247 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:30.032653 (+ 406us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:30.032658 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:30.034320 (+ 1662us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.40.126:44911: Server oryx/127.6.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":15}
W20260806 22:47:30.034446 18718 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:47:30.029788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.67:47407 (local address 127.6.40.126:44911)
0806 22:47:30.029847 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:30.029849 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:30.029873 (+ 24us) server_negotiation.cc:407] Connection header received
0806 22:47:30.029893 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:30.029894 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:30.029902 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:30.029969 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:30.030268 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:30.030961 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:30.031907 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:30.032098 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:30.032173 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:30.034386 (+ 2213us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.40.67:47407: BlockingRecv error: recv got EOF from 127.6.40.67:47407 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 06 22:47:30 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.67@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:47:30 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.67@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:47:30.038583 19078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:30.034711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:44911 (local address 127.6.40.67:49125)
0806 22:47:30.034812 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:30.034822 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:30.034865 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:30.034969 (+ 104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:30.034970 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:30.035075 (+ 105us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:30.035098 (+ 23us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:30.035099 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:30.035624 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:30.035625 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:30.036081 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:30.036082 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:30.036375 (+ 293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:30.036376 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:30.036411 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:30.036780 (+ 369us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:30.036784 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:30.038536 (+ 1752us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.40.126:44911: Server oryx/127.6.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":87}
W20260806 22:47:30.038699 18718 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:47:30.034768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.67:49125 (local address 127.6.40.126:44911)
0806 22:47:30.034822 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:30.034823 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:30.034854 (+ 31us) server_negotiation.cc:407] Connection header received
0806 22:47:30.034902 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:30.034902 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:30.034909 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:30.034954 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:30.035153 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:30.035565 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:30.036148 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:30.036313 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:30.036363 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:30.038627 (+ 2264us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.40.67:49125: BlockingRecv error: recv got EOF from 127.6.40.67:49125 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
W20260806 22:47:30.038724 19076 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 06 22:47:30 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.65@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:47:30 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.65@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:47:30.766108 19083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:30.760931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:44911 (local address 127.6.40.65:60247)
0806 22:47:30.761065 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:30.761084 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:30.761138 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:30.761271 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:30.761272 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:30.761487 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:30.761545 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:30.761547 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:30.762244 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:30.762246 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:30.762903 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:30.762907 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:30.763313 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:30.763315 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:30.763403 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:30.763992 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:30.763997 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:30.765987 (+ 1990us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.40.126:44911: Server oryx/127.6.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":74,"threads_started":1}
W20260806 22:47:30.766137 19085 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:47:30.761002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.65:60247 (local address 127.6.40.126:44911)
0806 22:47:30.761154 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:30.761156 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:30.761184 (+ 28us) server_negotiation.cc:407] Connection header received
0806 22:47:30.761205 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:30.761206 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:30.761216 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:30.761267 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:30.761613 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:30.762135 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:30.762998 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:30.763219 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:30.763276 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:30.766064 (+ 2788us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.40.65:60247: BlockingRecv error: recv got EOF from 127.6.40.65:60247 (error 108)
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":92,"threads_started":1}
Aug 06 22:47:30 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.66@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:47:30 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.66@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:47:30.899139 19090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:30.894012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:44911 (local address 127.6.40.66:48751)
0806 22:47:30.894176 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:30.894188 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:30.894257 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:30.894418 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:30.894418 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:30.894600 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:30.894654 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:30.894655 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:30.895461 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:30.895462 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:30.896006 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:30.896008 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:30.896505 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:30.896506 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:30.896601 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:30.897182 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:30.897188 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:30.899040 (+ 1852us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.40.126:44911: Server oryx/127.6.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":91,"threads_started":1}
W20260806 22:47:30.899209 19085 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:47:30.894061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.66:48751 (local address 127.6.40.126:44911)
0806 22:47:30.894134 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:30.894137 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:30.894265 (+ 128us) server_negotiation.cc:407] Connection header received
0806 22:47:30.894310 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:30.894311 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:30.894322 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:30.894401 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:30.894762 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:30.895374 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:30.896119 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:30.896415 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:30.896504 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:30.899142 (+ 2638us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.40.66:48751: BlockingRecv error: recv got EOF from 127.6.40.66:48751 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Aug 06 22:47:31 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.67@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:47:31 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.67@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:47:31.043695 19092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:31.039100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:44911 (local address 127.6.40.67:46905)
0806 22:47:31.039258 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:31.039269 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:31.039328 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:31.039490 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:31.039491 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:31.039681 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:31.039738 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:31.039740 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:31.040358 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:31.040358 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:31.040935 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:31.040937 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:31.041257 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:31.041258 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:31.041298 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:31.041752 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:31.041756 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:31.043625 (+ 1869us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.40.126:44911: Server oryx/127.6.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":92,"threads_started":1}
W20260806 22:47:31.043825 19085 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:47:31.039171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.67:46905 (local address 127.6.40.126:44911)
0806 22:47:31.039218 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:31.039219 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:31.039361 (+ 142us) server_negotiation.cc:407] Connection header received
0806 22:47:31.039396 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:31.039397 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:31.039412 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:31.039470 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:31.039805 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:31.040292 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:31.041010 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:31.041192 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:31.041245 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:31.043731 (+ 2486us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.40.67:46905: BlockingRecv error: recv got EOF from 127.6.40.67:46905 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
Aug 06 22:47:31 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.65@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:47:31 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.65@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:47:31.771570 19093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:31.766786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:44911 (local address 127.6.40.65:50115)
0806 22:47:31.766999 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:31.767033 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:31.767108 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:31.767280 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:31.767281 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:31.767505 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:31.767575 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:31.767577 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:31.768258 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:31.768258 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:31.768789 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:31.768791 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:31.769203 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:31.769204 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:31.769251 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:31.769698 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:31.769703 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:31.771499 (+ 1796us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.40.126:44911: Server oryx/127.6.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":79,"threads_started":1}
W20260806 22:47:31.771723 19094 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:47:31.766865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.65:50115 (local address 127.6.40.126:44911)
0806 22:47:31.767019 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:31.767021 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:31.767119 (+ 98us) server_negotiation.cc:407] Connection header received
0806 22:47:31.767177 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:31.767178 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:31.767193 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:31.767262 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:31.767648 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:31.768172 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:31.768923 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:31.769129 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:31.769188 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:31.771626 (+ 2438us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.40.65:50115: BlockingRecv error: recv got EOF from 127.6.40.65:50115 (error 108)
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":77,"threads_started":1}
Aug 06 22:47:31 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.66@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:47:31 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.66@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:47:31.905157 19096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:31.899634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:44911 (local address 127.6.40.66:52147)
0806 22:47:31.899857 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:31.899870 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:31.899973 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:31.900153 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:31.900154 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:31.900369 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:31.900438 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:31.900440 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:31.901229 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:31.901231 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:31.901818 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:31.901821 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:31.902251 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:31.902253 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:31.902312 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:31.902967 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:31.902972 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:31.905049 (+ 2077us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.40.126:44911: Server oryx/127.6.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":89,"threads_started":1}
W20260806 22:47:31.905213 19094 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:47:31.899737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.66:52147 (local address 127.6.40.126:44911)
0806 22:47:31.899817 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:31.899820 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:31.899947 (+ 127us) server_negotiation.cc:407] Connection header received
0806 22:47:31.900047 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:31.900048 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:31.900065 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:31.900145 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:31.900535 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:31.901136 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:31.901934 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:31.902144 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:31.902252 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:31.905140 (+ 2888us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.40.66:52147: BlockingRecv error: recv got EOF from 127.6.40.66:52147 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 06 22:47:32 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.67@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:47:32 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.67@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:47:32.049052 19097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:32.044414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:44911 (local address 127.6.40.67:47213)
0806 22:47:32.044605 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:32.044620 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:32.044692 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:32.044850 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:32.044850 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:32.045039 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:32.045097 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:32.045099 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:32.045734 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:32.045735 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:32.046286 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:32.046287 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:32.046600 (+ 313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:32.046601 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:32.046641 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:32.047080 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:32.047085 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:32.048945 (+ 1860us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.40.126:44911: Server oryx/127.6.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":103,"threads_started":1}
W20260806 22:47:32.049237 19094 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:47:32.044510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.67:47213 (local address 127.6.40.126:44911)
0806 22:47:32.044611 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:32.044613 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:32.044690 (+ 77us) server_negotiation.cc:407] Connection header received
0806 22:47:32.044750 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:32.044751 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:32.044768 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:32.044811 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:32.045167 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:32.045666 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:32.046358 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:32.046536 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:32.046585 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:32.049127 (+ 2542us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.40.67:47213: BlockingRecv error: recv got EOF from 127.6.40.67:47213 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Aug 06 22:47:32 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.65@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:47:32 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.65@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:47:32.780967 19099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:32.772005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:44911 (local address 127.6.40.65:46013)
0806 22:47:32.772161 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:32.772172 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:32.772221 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:32.772392 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:32.772393 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:32.772570 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:32.772627 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:32.772629 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:32.773379 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:32.773380 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:32.773940 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:32.773942 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:32.774289 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:32.774290 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:32.774335 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:32.778627 (+ 4292us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:32.778633 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:32.780825 (+ 2192us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.40.126:44911: Server oryx/127.6.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":88,"threads_started":1}
W20260806 22:47:32.781163 19101 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:47:32.772125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.65:46013 (local address 127.6.40.126:44911)
0806 22:47:32.772272 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:32.772273 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:32.772281 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:47:32.772303 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:32.772303 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:32.772314 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:32.772376 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:32.772706 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:32.773306 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:32.774022 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:32.774221 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:32.774274 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:32.781059 (+ 6785us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.40.65:46013: BlockingRecv error: recv got EOF from 127.6.40.65:46013 (error 108)
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":89,"threads_started":1}
Aug 06 22:47:32 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.66@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:47:32 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.66@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:47:32.910974 19102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:32.905706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:44911 (local address 127.6.40.66:52087)
0806 22:47:32.905923 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:32.905939 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:32.906031 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:32.906230 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:32.906231 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:32.906482 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:32.906543 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:32.906546 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:32.907274 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:32.907275 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:32.907857 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:32.907859 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:32.908276 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:32.908277 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:32.908351 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:32.908864 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:32.908870 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:32.910906 (+ 2036us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.40.126:44911: Server oryx/127.6.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":99,"threads_started":1}
W20260806 22:47:32.911080 19101 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:47:32.905838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.66:52087 (local address 127.6.40.126:44911)
0806 22:47:32.905923 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:32.905926 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:32.906006 (+ 80us) server_negotiation.cc:407] Connection header received
0806 22:47:32.906109 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:32.906110 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:32.906130 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:32.906201 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:32.906645 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:32.907177 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:32.907959 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:32.908182 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:32.908275 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:32.911010 (+ 2735us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.40.66:52087: BlockingRecv error: recv got EOF from 127.6.40.66:52087 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 06 22:47:33 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.67@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:47:33 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.67@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:47:33.054550 19104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:33.049643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:44911 (local address 127.6.40.67:45207)
0806 22:47:33.049799 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:33.049809 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:33.049868 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:33.050007 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:33.050008 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:33.050187 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:33.050245 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:33.050247 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:33.050931 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:33.050932 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:33.051500 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:33.051502 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:33.051854 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:33.051855 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:33.051902 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:33.052496 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:33.052501 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:33.054425 (+ 1924us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.40.126:44911: Server oryx/127.6.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":91,"threads_started":1}
W20260806 22:47:33.054584 19101 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:47:33.049746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.67:45207 (local address 127.6.40.126:44911)
0806 22:47:33.049796 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:33.049799 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:33.049851 (+ 52us) server_negotiation.cc:407] Connection header received
0806 22:47:33.049930 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:33.049931 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:33.049945 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:33.049987 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:33.050323 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:33.050847 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:33.051591 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:33.051781 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:33.051839 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:33.054508 (+ 2669us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.40.67:45207: BlockingRecv error: recv got EOF from 127.6.40.67:45207 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
Aug 06 22:47:33 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.65@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:47:33 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.65@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:47:33.786446 19105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:33.781527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:44911 (local address 127.6.40.65:36885)
0806 22:47:33.781681 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:33.781693 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:33.781757 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:33.781923 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:33.781923 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:33.782177 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:33.782236 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:33.782238 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:33.782985 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:33.782986 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:33.783571 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:33.783573 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:33.783972 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:33.783973 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:33.784024 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:33.784464 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:33.784468 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:33.786384 (+ 1916us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.40.126:44911: Server oryx/127.6.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":76,"threads_started":1}
W20260806 22:47:33.786581 19106 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:47:33.781569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.65:36885 (local address 127.6.40.126:44911)
0806 22:47:33.781729 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:33.781731 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:33.781742 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:47:33.781818 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:33.781819 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:33.781830 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:33.781916 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:33.782307 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:33.782895 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:33.783668 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:33.783891 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:33.783965 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:33.786493 (+ 2528us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.40.65:36885: BlockingRecv error: recv got EOF from 127.6.40.65:36885 (error 108)
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":83,"threads_started":1}
Aug 06 22:47:33 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.66@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:47:33 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.66@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:47:33.917269 19111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:33.911588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:44911 (local address 127.6.40.66:45545)
0806 22:47:33.911756 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:33.911770 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:33.911836 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:33.911989 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:33.911990 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:33.912177 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:33.912231 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:33.912233 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:33.912931 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:33.912932 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:33.913703 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:33.913707 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:33.914182 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:33.914184 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:33.914282 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:33.914903 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:33.914942 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:33.917182 (+ 2240us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.40.126:44911: Server oryx/127.6.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":113,"threads_started":1}
W20260806 22:47:33.917438 19106 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:47:33.911636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.66:45545 (local address 127.6.40.126:44911)
0806 22:47:33.911704 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:33.911706 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:33.911831 (+ 125us) server_negotiation.cc:407] Connection header received
0806 22:47:33.911891 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:33.911891 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:33.911901 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:33.911955 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:33.912292 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:33.912814 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:33.913825 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:33.914101 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:33.914191 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:33.917349 (+ 3158us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.40.66:45545: BlockingRecv error: recv got EOF from 127.6.40.66:45545 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 06 22:47:34 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.67@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:47:34 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.67@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:47:34.060170 19113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:34.055072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:44911 (local address 127.6.40.67:37549)
0806 22:47:34.055282 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:34.055295 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:34.055386 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:34.055576 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:34.055577 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:34.055772 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:34.055829 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:34.055831 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:34.056577 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:34.056579 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:34.057202 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:34.057205 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:34.057603 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:34.057604 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:34.057670 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:34.058138 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:34.058142 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:34.060110 (+ 1968us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.40.126:44911: Server oryx/127.6.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":131,"threads_started":1}
W20260806 22:47:34.060321 19106 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:47:34.055177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.67:37549 (local address 127.6.40.126:44911)
0806 22:47:34.055264 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:34.055267 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:34.055367 (+ 100us) server_negotiation.cc:407] Connection header received
0806 22:47:34.055459 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:34.055460 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:34.055476 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:34.055550 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:34.055911 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:34.056482 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:34.057290 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:34.057513 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:34.057590 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:34.060227 (+ 2637us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.40.67:37549: BlockingRecv error: recv got EOF from 127.6.40.67:37549 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 06 22:47:34 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.65@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:47:34 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.65@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:47:34.792022 19114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:34.787061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:44911 (local address 127.6.40.65:52843)
0806 22:47:34.787238 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:34.787253 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:34.787369 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:34.787555 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:34.787556 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:34.787748 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:34.787804 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:34.787806 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:34.788515 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:34.788516 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:34.789126 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:34.789128 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:34.789506 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:34.789507 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:34.789573 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:34.790019 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:34.790023 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:34.791958 (+ 1935us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.40.126:44911: Server oryx/127.6.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":98,"threads_started":1}
W20260806 22:47:34.792182 19115 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:47:34.787087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.65:52843 (local address 127.6.40.126:44911)
0806 22:47:34.787226 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:34.787228 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:34.787315 (+ 87us) server_negotiation.cc:407] Connection header received
0806 22:47:34.787447 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:34.787449 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:34.787463 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:34.787548 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:34.787885 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:34.788428 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:34.789210 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:34.789427 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:34.789507 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:34.792081 (+ 2574us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.40.65:52843: BlockingRecv error: recv got EOF from 127.6.40.65:52843 (error 108)
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":80,"threads_started":1}
Aug 06 22:47:34 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.66@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:47:34 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.66@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:47:34.923058 19116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:34.917934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:44911 (local address 127.6.40.66:39453)
0806 22:47:34.918151 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:34.918165 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:34.918240 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:34.918434 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:34.918436 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:34.918697 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:34.918756 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:34.918758 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:34.919456 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:34.919457 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:34.920046 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:34.920048 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:34.920441 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:34.920443 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:34.920507 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:34.921147 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:34.921152 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:34.922941 (+ 1789us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.40.126:44911: Server oryx/127.6.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":130,"threads_started":1}
W20260806 22:47:34.923115 19115 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:47:34.918070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.66:39453 (local address 127.6.40.126:44911)
0806 22:47:34.918137 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:34.918139 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:34.918250 (+ 111us) server_negotiation.cc:407] Connection header received
0806 22:47:34.918329 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:34.918330 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:34.918344 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:34.918418 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:34.918844 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:34.919362 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:34.920138 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:34.920352 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:34.920439 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:34.923044 (+ 2605us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.40.66:39453: BlockingRecv error: recv got EOF from 127.6.40.66:39453 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 06 22:47:35 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.67@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:47:35 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.67@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:47:35.065809 19118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:35.060738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:44911 (local address 127.6.40.67:43039)
0806 22:47:35.060909 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:35.060920 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:35.061037 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:35.061214 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:35.061215 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:35.061410 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:35.061467 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:35.061469 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:35.062162 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:35.062163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:35.062713 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:35.062715 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:35.063098 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:35.063099 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:35.063165 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:35.063635 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:35.063639 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:35.065717 (+ 2078us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.40.126:44911: Server oryx/127.6.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":77,"threads_started":1}
W20260806 22:47:35.065909 19115 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:47:35.060813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.67:43039 (local address 127.6.40.126:44911)
0806 22:47:35.060886 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:35.060889 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:35.061015 (+ 126us) server_negotiation.cc:407] Connection header received
0806 22:47:35.061102 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:35.061103 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:35.061119 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:35.061190 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:35.061561 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:35.062067 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:35.062802 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:35.063007 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:35.063095 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:35.065817 (+ 2722us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.40.67:43039: BlockingRecv error: recv got EOF from 127.6.40.67:43039 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 06 22:47:35 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.65@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:47:35 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.65@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:47:35.797857 19119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:35.792617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:44911 (local address 127.6.40.65:42135)
0806 22:47:35.792908 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:35.792921 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:35.792992 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:35.793170 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:35.793171 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:35.793350 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:35.793404 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:35.793405 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:35.794360 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:35.794361 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:35.794919 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:35.794921 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:35.795371 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:35.795372 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:35.795440 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:35.795927 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:35.795932 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:35.797796 (+ 1864us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.40.126:44911: Server oryx/127.6.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":188,"threads_started":1}
W20260806 22:47:35.797995 19120 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:47:35.792710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.65:42135 (local address 127.6.40.126:44911)
0806 22:47:35.792895 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:35.792900 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:35.792987 (+ 87us) server_negotiation.cc:407] Connection header received
0806 22:47:35.793072 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:35.793072 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:35.793085 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:35.793153 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:35.793472 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:35.794271 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:35.794988 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:35.795281 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:35.795350 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:35.797901 (+ 2551us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.40.65:42135: BlockingRecv error: recv got EOF from 127.6.40.65:42135 (error 108)
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":78,"threads_started":1}
Aug 06 22:47:35 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.66@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:47:35 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.66@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:47:35.929352 19121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:35.923629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:44911 (local address 127.6.40.66:50805)
0806 22:47:35.923779 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:35.923798 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:35.923876 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:35.924102 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:35.924103 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:35.924304 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:35.924363 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:35.924364 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:35.925263 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:35.925264 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:35.926111 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:35.926114 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:35.926564 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:35.926565 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:35.926611 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:35.927156 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:35.927161 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:35.929274 (+ 2113us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.40.126:44911: Server oryx/127.6.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":86,"threads_started":1}
W20260806 22:47:35.929560 19120 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:47:35.923807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.66:50805 (local address 127.6.40.126:44911)
0806 22:47:35.923881 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:35.923884 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:35.923931 (+ 47us) server_negotiation.cc:407] Connection header received
0806 22:47:35.923965 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:35.923966 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:35.923980 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:35.924052 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:35.924470 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:35.925142 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:35.926209 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:35.926475 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:35.926549 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:35.929462 (+ 2913us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.40.66:50805: BlockingRecv error: recv got EOF from 127.6.40.66:50805 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 06 22:47:36 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.67@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:47:36 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.67@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:47:36.072016 19124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:36.066410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:44911 (local address 127.6.40.67:43819)
0806 22:47:36.066593 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:36.066605 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:36.066680 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:36.066821 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:36.066821 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:36.067004 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:36.067060 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:36.067062 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:36.068054 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:36.068055 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:36.068639 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:36.068642 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:36.069014 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:36.069015 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:36.069068 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:36.069517 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:36.069522 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:36.071898 (+ 2376us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.40.126:44911: Server oryx/127.6.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":97,"threads_started":1}
W20260806 22:47:36.072082 19120 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:47:36.066474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.67:43819 (local address 127.6.40.126:44911)
0806 22:47:36.066548 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:36.066551 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:36.066676 (+ 125us) server_negotiation.cc:407] Connection header received
0806 22:47:36.066726 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:36.066727 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:36.066742 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:36.066796 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:36.067123 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:36.067956 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:36.068724 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:36.068943 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:36.069000 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:36.071999 (+ 2999us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.40.67:43819: BlockingRecv error: recv got EOF from 127.6.40.67:43819 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 06 22:47:36 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.65@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:47:36 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.65@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:47:36.811090 19125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:36.798491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:44911 (local address 127.6.40.65:42427)
0806 22:47:36.798670 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:36.798684 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:36.798761 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:36.798920 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:36.798921 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:36.799105 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:36.799166 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:36.799168 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:36.799835 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:36.799836 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:36.800378 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:36.800380 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:36.801000 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:36.801001 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:36.801117 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:36.808632 (+ 7515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:36.808639 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:36.810990 (+ 2351us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.40.126:44911: Server oryx/127.6.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":87,"threads_started":1}
W20260806 22:47:36.811177 19126 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:47:36.798575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.65:42427 (local address 127.6.40.126:44911)
0806 22:47:36.798760 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:36.798762 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:36.798778 (+ 16us) server_negotiation.cc:407] Connection header received
0806 22:47:36.798821 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:36.798822 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:36.798838 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:36.798902 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:36.799233 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:36.799760 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:36.800475 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:36.800827 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:36.800949 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:36.811100 (+ 10151us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.40.65:42427: BlockingRecv error: recv got EOF from 127.6.40.65:42427 (error 108)
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":120,"threads_started":1}
Aug 06 22:47:36 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.66@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:47:36 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.66@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:47:36.936489 19130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:36.929928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:44911 (local address 127.6.40.66:38707)
0806 22:47:36.930048 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:36.930062 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:36.930140 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:36.930322 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:36.930323 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:36.930590 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:36.930666 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:36.930669 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:36.931661 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:36.931662 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:36.932664 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:36.932668 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:36.933260 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:36.933261 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:36.933338 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:36.933961 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:36.933968 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:36.936393 (+ 2425us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.40.126:44911: Server oryx/127.6.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":67,"threads_started":1}
W20260806 22:47:36.936638 19126 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:47:36.929986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.66:38707 (local address 127.6.40.126:44911)
0806 22:47:36.930045 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:36.930047 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:36.930135 (+ 88us) server_negotiation.cc:407] Connection header received
0806 22:47:36.930228 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:36.930229 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:36.930245 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:36.930299 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:36.930747 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:36.931555 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:36.932769 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:36.933149 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:36.933228 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:36.936562 (+ 3334us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.40.66:38707: BlockingRecv error: recv got EOF from 127.6.40.66:38707 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
Aug 06 22:47:37 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.67@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:47:37 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.67@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:47:37.077699 19132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:37.072615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:44911 (local address 127.6.40.67:33797)
0806 22:47:37.072791 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:37.072803 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:37.072919 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:37.073106 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:37.073106 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:37.073294 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:37.073354 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:37.073356 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:37.074003 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:37.074004 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:37.074550 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:37.074551 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:37.074865 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:37.074866 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:37.074910 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:37.075393 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:37.075399 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:37.077637 (+ 2238us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.40.126:44911: Server oryx/127.6.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":120,"threads_started":1}
W20260806 22:47:37.077859 19126 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:47:37.072676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.67:33797 (local address 127.6.40.126:44911)
0806 22:47:37.072767 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:37.072769 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:37.072900 (+ 131us) server_negotiation.cc:407] Connection header received
0806 22:47:37.072994 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:37.072995 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:37.073010 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:37.073084 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:37.073424 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:37.073921 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:37.074624 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:37.074802 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:37.074851 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:37.077767 (+ 2916us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.40.67:33797: BlockingRecv error: recv got EOF from 127.6.40.67:33797 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 06 22:47:37 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.65@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:47:37 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.65@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:47:37.822741 19133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:37.811728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:44911 (local address 127.6.40.65:33959)
0806 22:47:37.811953 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:37.811968 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:37.812037 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:37.812224 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:37.812225 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:37.812421 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:37.812477 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:37.812479 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:37.813230 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:37.813231 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:37.813846 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:37.813849 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:37.814345 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:37.814346 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:37.814457 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:37.820735 (+ 6278us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:37.820739 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:37.822679 (+ 1940us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.40.126:44911: Server oryx/127.6.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":103,"threads_started":1}
W20260806 22:47:37.822889 19134 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:47:37.811891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.65:33959 (local address 127.6.40.126:44911)
0806 22:47:37.812058 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:37.812061 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:37.812071 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:47:37.812098 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:37.812099 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:37.812112 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:37.812204 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:37.812555 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:37.813108 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:37.813923 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:37.814263 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:37.814334 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:37.822790 (+ 8456us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.40.65:33959: BlockingRecv error: recv got EOF from 127.6.40.65:33959 (error 108)
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":72,"threads_started":1}
Aug 06 22:47:37 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.66@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:47:37 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.66@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:47:37.942381 19136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:37.937194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:44911 (local address 127.6.40.66:49553)
0806 22:47:37.937402 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:37.937414 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:37.937490 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:37.937630 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:37.937631 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:37.937825 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:37.937887 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:37.937889 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:37.938703 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:37.938705 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:37.939270 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:37.939274 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:37.939826 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:37.939827 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:37.939881 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:37.940363 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:37.940368 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:37.942275 (+ 1907us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.40.126:44911: Server oryx/127.6.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":105,"threads_started":1}
W20260806 22:47:37.942425 19134 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:47:37.937253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.66:49553 (local address 127.6.40.126:44911)
0806 22:47:37.937335 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:37.937338 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:37.937487 (+ 149us) server_negotiation.cc:407] Connection header received
0806 22:47:37.937543 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:37.937544 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:37.937554 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:37.937607 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:37.937956 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:37.938610 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:37.939571 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:37.939755 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:37.939805 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:37.942361 (+ 2556us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.40.66:49553: BlockingRecv error: recv got EOF from 127.6.40.66:49553 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 06 22:47:38 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.67@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:47:38 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.67@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:47:38.083112 19137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:38.078187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:44911 (local address 127.6.40.67:39355)
0806 22:47:38.078349 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:38.078363 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:38.078436 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:38.078553 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:38.078554 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:38.078751 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:38.078810 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:38.078812 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:38.079472 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:38.079474 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:38.080079 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:38.080081 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:38.080408 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:38.080409 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:38.080458 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:38.080996 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:38.081001 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:38.082993 (+ 1992us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.40.126:44911: Server oryx/127.6.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":97,"threads_started":1}
W20260806 22:47:38.083168 19134 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:47:38.078266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.67:39355 (local address 127.6.40.126:44911)
0806 22:47:38.078329 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:38.078331 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:38.078435 (+ 104us) server_negotiation.cc:407] Connection header received
0806 22:47:38.078483 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:38.078483 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:38.078495 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:38.078547 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:38.078889 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:38.079395 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:38.080159 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:38.080336 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:38.080385 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:38.083077 (+ 2692us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.40.67:39355: BlockingRecv error: recv got EOF from 127.6.40.67:39355 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Aug 06 22:47:38 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.65@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:47:38 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.65@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:47:38.828485 19138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:38.823279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:44911 (local address 127.6.40.65:51387)
0806 22:47:38.823434 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:38.823446 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:38.823495 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:38.823662 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:38.823663 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:38.823862 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:38.823913 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:38.823915 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:38.824637 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:38.824638 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:38.825259 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:38.825262 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:38.825656 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:38.825657 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:38.825699 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:38.826153 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:38.826157 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:38.828407 (+ 2250us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.40.126:44911: Server oryx/127.6.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":85,"threads_started":1}
W20260806 22:47:38.828649 19141 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:47:38.823381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.65:51387 (local address 127.6.40.126:44911)
0806 22:47:38.823527 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:38.823528 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:38.823536 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:47:38.823559 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:38.823559 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:38.823571 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:38.823638 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:38.823994 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:38.824564 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:38.825376 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:38.825580 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:38.825644 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:38.828539 (+ 2895us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.40.65:51387: BlockingRecv error: recv got EOF from 127.6.40.65:51387 (error 108)
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":85,"threads_started":1}
Aug 06 22:47:38 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.66@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:47:38 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.66@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:47:38.948282 19142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:38.942966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:44911 (local address 127.6.40.66:43399)
0806 22:47:38.943183 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:38.943198 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:38.943287 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:38.943435 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:38.943436 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:38.943632 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:38.943690 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:38.943691 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:38.944411 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:38.944412 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:38.944994 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:38.944996 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:38.945385 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:38.945386 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:38.945455 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:38.945985 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:38.945990 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:38.948181 (+ 2191us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.40.126:44911: Server oryx/127.6.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":109,"threads_started":1}
W20260806 22:47:38.948338 19141 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:47:38.943111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.66:43399 (local address 127.6.40.126:44911)
0806 22:47:38.943207 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:38.943209 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:38.943277 (+ 68us) server_negotiation.cc:407] Connection header received
0806 22:47:38.943349 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:38.943350 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:38.943364 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:38.943417 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:38.943789 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:38.944308 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:38.945092 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:38.945304 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:38.945386 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:38.948268 (+ 2882us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.40.66:43399: BlockingRecv error: recv got EOF from 127.6.40.66:43399 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 06 22:47:39 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.67@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:47:39 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.67@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:47:39.088898 19143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:39.083768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:44911 (local address 127.6.40.67:58167)
0806 22:47:39.083958 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:39.083971 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:39.084072 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:39.084244 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:39.084245 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:39.084446 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:39.084500 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:39.084502 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:39.085285 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:39.085287 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:39.085828 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:39.085830 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:39.086207 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:39.086208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:39.086279 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:39.086774 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:39.086779 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:39.088748 (+ 1969us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.40.126:44911: Server oryx/127.6.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":95,"threads_started":1}
W20260806 22:47:39.088930 19141 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:47:39.083877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.67:58167 (local address 127.6.40.126:44911)
0806 22:47:39.083956 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:39.083958 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:39.084051 (+ 93us) server_negotiation.cc:407] Connection header received
0806 22:47:39.084139 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:39.084140 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:39.084163 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:39.084232 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:39.084604 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:39.085189 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:39.085916 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:39.086117 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:39.086204 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:39.088864 (+ 2660us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.40.67:58167: BlockingRecv error: recv got EOF from 127.6.40.67:58167 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 06 22:47:39 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.65@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:47:39 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.65@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:47:39.834112 19144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:39.829010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:44911 (local address 127.6.40.65:34011)
0806 22:47:39.829225 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:39.829239 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:39.829315 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:39.829537 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:39.829538 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:39.829806 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:39.829900 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:39.829903 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:39.830625 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:39.830626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:39.831190 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:39.831192 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:39.831538 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:39.831539 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:39.831589 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:39.832146 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:39.832151 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:39.834050 (+ 1899us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.40.126:44911: Server oryx/127.6.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":123,"threads_started":1}
W20260806 22:47:39.834308 19146 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:47:39.829112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.65:34011 (local address 127.6.40.126:44911)
0806 22:47:39.829227 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:39.829229 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:39.829309 (+ 80us) server_negotiation.cc:407] Connection header received
0806 22:47:39.829417 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:39.829418 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:39.829434 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:39.829506 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:39.829983 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:39.830496 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:39.831268 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:39.831465 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:39.831525 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:39.834200 (+ 2675us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.40.65:34011: BlockingRecv error: recv got EOF from 127.6.40.65:34011 (error 108)
Metrics: {"server-negotiator.queue_time_us":89,"thread_start_us":53,"threads_started":1}
Aug 06 22:47:39 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.66@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:47:39 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.66@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:47:39.953986 19147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:39.949002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:44911 (local address 127.6.40.66:44123)
0806 22:47:39.949210 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:39.949237 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:39.949313 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:39.949455 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:39.949456 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:39.949685 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:39.949743 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:39.949745 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:39.950404 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:39.950405 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:39.950936 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:39.950937 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:39.951287 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:39.951288 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:39.951330 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:39.951803 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:39.951808 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:39.953920 (+ 2112us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.40.126:44911: Server oryx/127.6.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":75,"threads_started":1}
W20260806 22:47:39.954228 19146 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:47:39.949095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.66:44123 (local address 127.6.40.126:44911)
0806 22:47:39.949155 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:39.949158 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:39.949320 (+ 162us) server_negotiation.cc:407] Connection header received
0806 22:47:39.949366 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:39.949367 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:39.949381 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:39.949452 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:39.949822 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:39.950330 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:39.951019 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:39.951216 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:39.951275 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:39.954180 (+ 2905us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.40.66:44123: BlockingRecv error: recv got EOF from 127.6.40.66:44123 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
Aug 06 22:47:40 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.67@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:47:40 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.67@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:47:40.094761 19148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:40.089641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:44911 (local address 127.6.40.67:47127)
0806 22:47:40.089823 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:40.089837 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:40.089906 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:40.090090 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:40.090091 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:40.090280 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:40.090345 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:40.090347 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:40.091201 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:40.091202 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:40.091776 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:40.091778 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:40.092123 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:40.092123 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:40.092166 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:40.092702 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:40.092708 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:40.094701 (+ 1993us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.40.126:44911: Server oryx/127.6.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":112,"threads_started":1}
W20260806 22:47:40.094900 19146 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:47:40.089777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.67:47127 (local address 127.6.40.126:44911)
0806 22:47:40.089834 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:40.089836 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:40.089887 (+ 51us) server_negotiation.cc:407] Connection header received
0806 22:47:40.089962 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:40.089963 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:40.089973 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:40.090044 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:40.090437 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:40.091118 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:40.091854 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:40.092051 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:40.092105 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:40.094814 (+ 2709us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.40.67:47127: BlockingRecv error: recv got EOF from 127.6.40.67:47127 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
Aug 06 22:47:40 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.65@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:47:40 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.65@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:47:40.840157 19150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:40.834685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:44911 (local address 127.6.40.65:32831)
0806 22:47:40.834851 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:40.834865 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:40.834965 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:40.835145 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:40.835146 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:40.835352 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:40.835408 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:40.835410 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:40.836066 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:40.836067 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:40.836638 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:40.836640 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:40.837111 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:40.837113 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:40.837165 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:40.837730 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:40.837735 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:40.840086 (+ 2351us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.40.126:44911: Server oryx/127.6.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":78,"threads_started":1}
W20260806 22:47:40.840304 19151 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:47:40.834762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.65:32831 (local address 127.6.40.126:44911)
0806 22:47:40.834905 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:40.834907 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:40.834940 (+ 33us) server_negotiation.cc:407] Connection header received
0806 22:47:40.835036 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:40.835037 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:40.835052 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:40.835119 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:40.835480 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:40.835995 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:40.836738 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:40.837026 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:40.837094 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:40.840189 (+ 3095us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.40.65:32831: BlockingRecv error: recv got EOF from 127.6.40.65:32831 (error 108)
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":79,"threads_started":1}
Aug 06 22:47:40 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.66@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:47:40 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.66@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:47:40.960022 19152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:40.954574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:44911 (local address 127.6.40.66:35043)
0806 22:47:40.954710 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:40.954724 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:40.954802 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:40.954988 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:40.954990 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:40.955211 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:40.955264 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:40.955266 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:40.956074 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:40.956076 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:40.956783 (+ 707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:40.956785 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:40.957271 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:40.957272 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:40.957343 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:40.957854 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:40.957858 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:40.959909 (+ 2051us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.40.126:44911: Server oryx/127.6.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":75,"threads_started":1}
W20260806 22:47:40.960072 19151 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:47:40.954652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.66:35043 (local address 127.6.40.126:44911)
0806 22:47:40.954711 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:40.954713 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:40.954777 (+ 64us) server_negotiation.cc:407] Connection header received
0806 22:47:40.954883 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:40.954883 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:40.954899 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:40.954981 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:40.955364 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:40.955967 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:40.956951 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:40.957173 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:40.957270 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:40.960007 (+ 2737us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.40.66:35043: BlockingRecv error: recv got EOF from 127.6.40.66:35043 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
Aug 06 22:47:41 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.67@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:47:41 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.67@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:47:41.100384 19155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:41.095288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:44911 (local address 127.6.40.67:55273)
0806 22:47:41.095473 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:41.095485 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:41.095554 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:41.095691 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:41.095691 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:41.095903 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:41.095960 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:41.095962 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:41.096761 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:41.096762 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:41.097459 (+ 697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:41.097463 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:41.097797 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:41.097797 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:41.097839 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:41.098403 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:41.098408 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:41.100294 (+ 1886us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.40.126:44911: Server oryx/127.6.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":87,"threads_started":1}
W20260806 22:47:41.100449 19151 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:47:41.095371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.67:55273 (local address 127.6.40.126:44911)
0806 22:47:41.095475 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:41.095477 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:41.095558 (+ 81us) server_negotiation.cc:407] Connection header received
0806 22:47:41.095599 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:41.095600 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:41.095612 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:41.095664 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:41.096044 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:41.096678 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:41.097546 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:41.097731 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:41.097787 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:41.100396 (+ 2609us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.40.67:55273: BlockingRecv error: recv got EOF from 127.6.40.67:55273 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 06 22:47:41 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.65@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:47:41 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.65@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:47:41.845537 19156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:41.840721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:44911 (local address 127.6.40.65:49949)
0806 22:47:41.840902 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:41.840919 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:41.840996 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:41.841146 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:41.841147 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:41.841382 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:41.841437 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:41.841439 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:41.842100 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:41.842100 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:41.842676 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:41.842679 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:41.843010 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:41.843011 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:41.843056 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:41.843583 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:41.843588 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:41.845444 (+ 1856us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.40.126:44911: Server oryx/127.6.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":80,"threads_started":1}
W20260806 22:47:41.845674 19160 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:47:41.840876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.65:49949 (local address 127.6.40.126:44911)
0806 22:47:41.840998 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:41.841000 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:41.841008 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:47:41.841049 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:41.841050 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:41.841061 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:41.841121 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:41.841510 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:41.842029 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:41.842752 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:41.842943 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:41.842993 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:41.845566 (+ 2573us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.40.65:49949: BlockingRecv error: recv got EOF from 127.6.40.65:49949 (error 108)
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":72,"threads_started":1}
Aug 06 22:47:41 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.66@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:47:41 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.66@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:47:41.965849 19161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:41.960591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:44911 (local address 127.6.40.66:57709)
0806 22:47:41.960771 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:41.960785 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:41.960912 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:41.961107 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:41.961107 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:41.961306 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:41.961369 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:41.961371 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:41.962086 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:41.962088 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:41.962655 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:41.962657 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:41.963069 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:41.963071 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:41.963137 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:41.963642 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:41.963647 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:41.965776 (+ 2129us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.40.126:44911: Server oryx/127.6.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":103,"threads_started":1}
W20260806 22:47:41.965988 19160 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:47:41.960701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.66:57709 (local address 127.6.40.126:44911)
0806 22:47:41.960775 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:41.960777 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:41.960877 (+ 100us) server_negotiation.cc:407] Connection header received
0806 22:47:41.960996 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:41.960997 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:41.961009 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:41.961085 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:41.961454 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:41.961978 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:41.962748 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:41.962970 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:41.963067 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:41.965886 (+ 2819us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.40.66:57709: BlockingRecv error: recv got EOF from 127.6.40.66:57709 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 06 22:47:42 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.67@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:47:42 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.67@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:47:42.105814 19162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:42.100976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:44911 (local address 127.6.40.67:35503)
0806 22:47:42.101154 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:42.101166 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:42.101235 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:42.101376 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:42.101377 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:42.101581 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:42.101638 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:42.101640 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:42.102294 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:42.102294 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:42.102830 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:42.102831 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:42.103186 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:42.103187 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:42.103229 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:42.103699 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:42.103704 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:42.105740 (+ 2036us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.40.126:44911: Server oryx/127.6.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":121,"threads_started":1}
W20260806 22:47:42.105985 19160 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:47:42.101050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.67:35503 (local address 127.6.40.126:44911)
0806 22:47:42.101139 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:42.101141 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:42.101239 (+ 98us) server_negotiation.cc:407] Connection header received
0806 22:47:42.101292 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:42.101293 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:42.101306 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:42.101359 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:42.101705 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:42.102224 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:42.102923 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:42.103114 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:42.103171 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:42.105904 (+ 2733us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.40.67:35503: BlockingRecv error: recv got EOF from 127.6.40.67:35503 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 06 22:47:42 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.65@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:47:42 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.65@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:47:42.856715 19165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:42.846112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:44911 (local address 127.6.40.65:50103)
0806 22:47:42.846299 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:42.846320 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:42.846392 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:42.846538 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:42.846539 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:42.846747 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:42.846818 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:42.846820 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:42.847613 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:42.847615 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:42.848608 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:42.848617 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:42.849164 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:42.849165 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:42.849229 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:42.854306 (+ 5077us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:42.854311 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:42.856645 (+ 2334us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.40.126:44911: Server oryx/127.6.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":119,"threads_started":1}
W20260806 22:47:42.856950 19166 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:47:42.846188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.65:50103 (local address 127.6.40.126:44911)
0806 22:47:42.846344 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:42.846346 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:42.846379 (+ 33us) server_negotiation.cc:407] Connection header received
0806 22:47:42.846441 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:42.846442 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:42.846457 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:42.846524 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:42.846936 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:42.847523 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:42.848717 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:42.849069 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:42.849148 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:42.856777 (+ 7629us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.40.65:50103: BlockingRecv error: recv got EOF from 127.6.40.65:50103 (error 108)
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":91,"threads_started":1}
Aug 06 22:47:42 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.66@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:47:42 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.66@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:47:42.971658 19168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:42.966424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:44911 (local address 127.6.40.66:46791)
0806 22:47:42.966595 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:42.966607 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:42.966663 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:42.966794 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:42.966794 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:42.967009 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:42.967065 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:42.967067 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:42.967785 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:42.967787 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:42.968518 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:42.968521 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:42.969057 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:42.969058 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:42.969110 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:42.969576 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:42.969581 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:42.971591 (+ 2010us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.40.126:44911: Server oryx/127.6.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":104,"threads_started":1}
W20260806 22:47:42.971792 19166 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:47:42.966607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.66:46791 (local address 127.6.40.126:44911)
0806 22:47:42.966675 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:42.966677 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:42.966693 (+ 16us) server_negotiation.cc:407] Connection header received
0806 22:47:42.966721 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:42.966722 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:42.966733 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:42.966786 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:42.967170 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:42.967694 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:42.968644 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:42.968991 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:42.969059 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:42.971707 (+ 2648us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.40.66:46791: BlockingRecv error: recv got EOF from 127.6.40.66:46791 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 06 22:47:43 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.67@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:47:43 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.67@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:47:43.111896 19169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:43.106301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:44911 (local address 127.6.40.67:43607)
0806 22:47:43.106456 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:43.106469 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:43.106536 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:43.106687 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:43.106688 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:43.106911 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:43.106968 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:43.106971 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:43.107666 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:43.107667 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:43.108246 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:43.108248 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:43.108625 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:43.108626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:43.108686 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:43.109325 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:43.109332 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:43.111764 (+ 2432us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.40.126:44911: Server oryx/127.6.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":91,"threads_started":1}
W20260806 22:47:43.111961 19166 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:47:43.106431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.67:43607 (local address 127.6.40.126:44911)
0806 22:47:43.106513 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:43.106514 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:43.106524 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:47:43.106596 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:43.106597 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:43.106610 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:43.106662 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:43.107038 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:43.107530 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:43.108325 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:43.108523 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:43.108587 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:43.111874 (+ 3287us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.40.67:43607: BlockingRecv error: recv got EOF from 127.6.40.67:43607 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 06 22:47:43 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.65@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:47:43 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.65@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:47:43.862120 19170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:43.857278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:44911 (local address 127.6.40.65:50651)
0806 22:47:43.857421 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:43.857432 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:43.857491 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:43.857639 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:43.857640 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:43.857813 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:43.857868 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:43.857870 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:43.858535 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:43.858536 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:43.859086 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:43.859087 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:43.859418 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:43.859419 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:43.859462 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:43.859981 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:43.859988 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:43.862058 (+ 2070us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.40.126:44911: Server oryx/127.6.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":90,"threads_started":1}
W20260806 22:47:43.862238 19171 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:47:43.857340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.65:50651 (local address 127.6.40.126:44911)
0806 22:47:43.857495 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:43.857497 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:43.857507 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:47:43.857530 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:43.857531 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:43.857541 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:43.857608 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:43.857956 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:43.858465 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:43.859159 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:43.859348 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:43.859398 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:43.862182 (+ 2784us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.40.65:50651: BlockingRecv error: recv got EOF from 127.6.40.65:50651 (error 108)
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":91,"threads_started":1}
Aug 06 22:47:43 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.66@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:47:43 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.66@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:47:43.976863 19172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:43.972222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:44911 (local address 127.6.40.66:56173)
0806 22:47:43.972430 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:43.972442 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:43.972507 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:43.972646 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:43.972647 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:43.972858 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:43.972912 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:43.972914 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:43.973572 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:43.973573 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:43.974109 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:43.974111 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:43.974439 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:43.974440 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:43.974480 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:43.974959 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:43.974964 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:43.976779 (+ 1815us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.40.126:44911: Server oryx/127.6.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":126,"threads_started":1}
W20260806 22:47:43.977121 19171 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:47:43.972283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.66:56173 (local address 127.6.40.126:44911)
0806 22:47:43.972369 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:43.972375 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:43.972507 (+ 132us) server_negotiation.cc:407] Connection header received
0806 22:47:43.972578 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:43.972578 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:43.972589 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:43.972641 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:43.972970 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:43.973496 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:43.974184 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:43.974370 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:43.974438 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:43.977072 (+ 2634us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.40.66:56173: BlockingRecv error: recv got EOF from 127.6.40.66:56173 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 06 22:47:44 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.67@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:47:44 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.67@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:47:44.117329 19173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:44.112368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:44911 (local address 127.6.40.67:37377)
0806 22:47:44.112517 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:44.112528 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:44.112606 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:44.112737 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:44.112738 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:44.113002 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:44.113073 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:44.113076 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:44.113748 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:44.113749 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:44.114407 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:44.114410 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:44.114733 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:44.114733 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:44.114775 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:44.115347 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:44.115351 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:44.117256 (+ 1905us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.40.126:44911: Server oryx/127.6.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":83,"threads_started":1}
W20260806 22:47:44.117455 19171 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:47:44.112444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.67:37377 (local address 127.6.40.126:44911)
0806 22:47:44.112524 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:44.112527 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:44.112600 (+ 73us) server_negotiation.cc:407] Connection header received
0806 22:47:44.112655 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:44.112656 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:44.112673 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:44.112734 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:44.113155 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:44.113670 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:44.114478 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:44.114668 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:44.114721 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:44.117350 (+ 2629us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.40.67:37377: BlockingRecv error: recv got EOF from 127.6.40.67:37377 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 06 22:47:44 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.65@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:47:44 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.65@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:47:44.867641 19174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:44.862843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:44911 (local address 127.6.40.65:43797)
0806 22:47:44.863018 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:44.863031 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:44.863094 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:44.863237 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:44.863238 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:44.863449 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:44.863511 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:44.863513 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:44.864180 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:44.864181 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:44.864791 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:44.864792 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:44.865220 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:44.865221 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:44.865268 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:44.865744 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:44.865748 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:44.867575 (+ 1827us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.40.126:44911: Server oryx/127.6.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":90,"threads_started":1}
W20260806 22:47:44.867758 19175 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:47:44.862945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.65:43797 (local address 127.6.40.126:44911)
0806 22:47:44.863115 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:44.863117 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:44.863126 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:47:44.863149 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:44.863149 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:44.863159 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:44.863218 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:44.863581 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:44.864098 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:44.864953 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:44.865152 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:44.865204 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:44.867678 (+ 2474us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.40.65:43797: BlockingRecv error: recv got EOF from 127.6.40.65:43797 (error 108)
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":80,"threads_started":1}
Aug 06 22:47:44 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.66@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:47:44 dist-test-slave-zh2d krb5kdc[17732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.40.66@KRBTEST.COM for oryx/127.6.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:47:44.982508 19177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:44.977592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:44911 (local address 127.6.40.66:40879)
0806 22:47:44.977753 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:44.977771 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:44.977837 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:44.978006 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:44.978007 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:44.978259 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:44.978330 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:44.978333 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:44.979055 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:44.979056 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:44.979604 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:44.979606 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:44.979937 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:44.979938 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:44.979977 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:44.980460 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:44.980464 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:44.982434 (+ 1970us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.40.126:44911: Server oryx/127.6.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":95,"threads_started":1}
W20260806 22:47:44.982609 19175 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:47:44.977696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.66:40879 (local address 127.6.40.126:44911)
0806 22:47:44.977749 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:44.977751 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:44.977819 (+ 68us) server_negotiation.cc:407] Connection header received
0806 22:47:44.977897 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:44.977898 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:44.977911 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:44.977983 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:44.978422 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:44.978982 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:44.979681 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:44.979871 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:44.979921 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:44.982538 (+ 2617us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.40.66:40879: BlockingRecv error: recv got EOF from 127.6.40.66:40879 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 22:47:45.031546 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 18517
I20260806 22:47:45.043504 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 18720
I20260806 22:47:45.057055 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 18896
I20260806 22:47:45.067206 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 18423
2026-08-06T22:47:45Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17738 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-06T22:47:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:47:45Z Disabled control of system clock
I20260806 22:47:45.102130 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056405724609-6305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056405724609-6305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056405724609-6305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.126:40715
--webserver_interface=127.6.40.126
--webserver_port=0
--builtin_ntp_servers=127.6.40.84:37609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.40.126:40715
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260806 22:47:45.187590 19182 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:47:45.187793 19182 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:47:45.187814 19182 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:47:45.189092 19182 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:47:45.189136 19182 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:47:45.189148 19182 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:47:45.189162 19182 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:47:45.189213 19182 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:47:45.190653 19182 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:37609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056405724609-6305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056405724609-6305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.40.126:40715
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.126:40715
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19182
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056405724609-6305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:47:45.190934 19182 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:47:45.191270 19182 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:47:45.196978 19190 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:45.197008 19182 server_base.cc:1034] running on GCE node
W20260806 22:47:45.196978 19193 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:45.197052 19189 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:45.197489 19182 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:47:45.197835 19182 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:47:45.199004 19182 hybrid_clock.cc:648] HybridClock initialized: now 1786056465198980 us; error 50 us; skew 500 ppm
I20260806 22:47:45.200284 19182 webserver.cc:460] Webserver started at http://127.6.40.126:46361/ using document root <none> and password file <none>
I20260806 22:47:45.200552 19182 fs_manager.cc:359] Metadata directory not provided
I20260806 22:47:45.200629 19182 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:47:45.200783 19182 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:47:45.201984 19182 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056405724609-6305-0/minicluster-data/master-0/data/instance:
uuid: "b727570ae8664d178671d36f01ac7f38"
format_stamp: "Formatted at 2026-08-06 22:47:45 on dist-test-slave-zh2d"
server_key: "34e7e73edac2959611fd4f4577e52189"
server_key_iv: "8c97ad4b2275fc00d1e51c63d9848832"
server_key_version: "encryptionkey@0"
I20260806 22:47:45.202364 19182 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056405724609-6305-0/minicluster-data/master-0/wal/instance:
uuid: "b727570ae8664d178671d36f01ac7f38"
format_stamp: "Formatted at 2026-08-06 22:47:45 on dist-test-slave-zh2d"
server_key: "34e7e73edac2959611fd4f4577e52189"
server_key_iv: "8c97ad4b2275fc00d1e51c63d9848832"
server_key_version: "encryptionkey@0"
I20260806 22:47:45.203698 19182 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:47:45.204430 19205 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:45.204604 19182 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:47:45.204710 19182 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056405724609-6305-0/minicluster-data/master-0/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056405724609-6305-0/minicluster-data/master-0/wal
uuid: "b727570ae8664d178671d36f01ac7f38"
format_stamp: "Formatted at 2026-08-06 22:47:45 on dist-test-slave-zh2d"
server_key: "34e7e73edac2959611fd4f4577e52189"
server_key_iv: "8c97ad4b2275fc00d1e51c63d9848832"
server_key_version: "encryptionkey@0"
I20260806 22:47:45.204795 19182 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056405724609-6305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056405724609-6305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056405724609-6305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:47:45.222507 19182 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:47:45.222862 19182 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:47:45.223021 19182 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:47:45.227487 19182 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.126:40715
I20260806 22:47:45.227535 19279 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.126:40715 every 8 connection(s)
I20260806 22:47:45.227905 19182 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
I20260806 22:47:45.228736 19281 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:47:45.231571 19281 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b727570ae8664d178671d36f01ac7f38: Bootstrap starting.
I20260806 22:47:45.232223 19281 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b727570ae8664d178671d36f01ac7f38: Neither blocks nor log segments found. Creating new log.
I20260806 22:47:45.232550 19281 log.cc:824] T 00000000000000000000000000000000 P b727570ae8664d178671d36f01ac7f38: Log is configured to *not* fsync() on all Append() calls
I20260806 22:47:45.233309 19281 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b727570ae8664d178671d36f01ac7f38: No bootstrap required, opened a new log
I20260806 22:47:45.234902 19281 raft_consensus.cc:348] T 00000000000000000000000000000000 P b727570ae8664d178671d36f01ac7f38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b727570ae8664d178671d36f01ac7f38" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 40715 } }
I20260806 22:47:45.235037 19281 raft_consensus.cc:374] T 00000000000000000000000000000000 P b727570ae8664d178671d36f01ac7f38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:47:45.235060 19281 raft_consensus.cc:729] T 00000000000000000000000000000000 P b727570ae8664d178671d36f01ac7f38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b727570ae8664d178671d36f01ac7f38, State: Initialized, Role: FOLLOWER
I20260806 22:47:45.235163 19281 consensus_queue.cc:260] T 00000000000000000000000000000000 P b727570ae8664d178671d36f01ac7f38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b727570ae8664d178671d36f01ac7f38" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 40715 } }
I20260806 22:47:45.235256 19281 raft_consensus.cc:388] T 00000000000000000000000000000000 P b727570ae8664d178671d36f01ac7f38 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:47:45.235284 19281 raft_consensus.cc:482] T 00000000000000000000000000000000 P b727570ae8664d178671d36f01ac7f38 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:47:45.235328 19281 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b727570ae8664d178671d36f01ac7f38 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:47:45.235877 19281 raft_consensus.cc:504] T 00000000000000000000000000000000 P b727570ae8664d178671d36f01ac7f38 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b727570ae8664d178671d36f01ac7f38" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 40715 } }
I20260806 22:47:45.235961 19281 leader_election.cc:302] T 00000000000000000000000000000000 P b727570ae8664d178671d36f01ac7f38 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b727570ae8664d178671d36f01ac7f38; no voters:
I20260806 22:47:45.236132 19281 leader_election.cc:288] T 00000000000000000000000000000000 P b727570ae8664d178671d36f01ac7f38 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:47:45.236225 19284 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b727570ae8664d178671d36f01ac7f38 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:47:45.236405 19281 sys_catalog.cc:564] T 00000000000000000000000000000000 P b727570ae8664d178671d36f01ac7f38 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:47:45.236410 19284 raft_consensus.cc:686] T 00000000000000000000000000000000 P b727570ae8664d178671d36f01ac7f38 [term 1 LEADER]: Becoming Leader. State: Replica: b727570ae8664d178671d36f01ac7f38, State: Running, Role: LEADER
I20260806 22:47:45.236619 19284 consensus_queue.cc:237] T 00000000000000000000000000000000 P b727570ae8664d178671d36f01ac7f38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b727570ae8664d178671d36f01ac7f38" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 40715 } }
I20260806 22:47:45.237164 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 19182
I20260806 22:47:45.237192 19286 sys_catalog.cc:455] T 00000000000000000000000000000000 P b727570ae8664d178671d36f01ac7f38 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b727570ae8664d178671d36f01ac7f38. Latest consensus state: current_term: 1 leader_uuid: "b727570ae8664d178671d36f01ac7f38" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b727570ae8664d178671d36f01ac7f38" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 40715 } } }
I20260806 22:47:45.237273 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056405724609-6305-0/minicluster-data/master-0/wal/instance
I20260806 22:47:45.237273 19286 sys_catalog.cc:458] T 00000000000000000000000000000000 P b727570ae8664d178671d36f01ac7f38 [sys.catalog]: This master's current role is: LEADER
I20260806 22:47:45.237514 6305 external_mini_cluster.cc:1428] Setting key 1ecdcd14f0e8bfbc3bd7656f5dcf0ba3
I20260806 22:47:45.237872 19285 sys_catalog.cc:455] T 00000000000000000000000000000000 P b727570ae8664d178671d36f01ac7f38 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b727570ae8664d178671d36f01ac7f38" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b727570ae8664d178671d36f01ac7f38" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 40715 } } }
I20260806 22:47:45.238005 19295 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:47:45.238011 19285 sys_catalog.cc:458] T 00000000000000000000000000000000 P b727570ae8664d178671d36f01ac7f38 [sys.catalog]: This master's current role is: LEADER
I20260806 22:47:45.238667 19295 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:47:45.240331 19295 catalog_manager.cc:1347] Generated new cluster ID: 3004c83395094df4b2063acb50c96fb4
I20260806 22:47:45.240386 19295 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:47:45.243798 19312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:45.238239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41234 (local address 127.6.40.126:40715)
0806 22:47:45.238455 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:45.238464 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:45.238600 (+ 136us) server_negotiation.cc:407] Connection header received
0806 22:47:45.238725 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:45.238732 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:45.238807 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:45.238961 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:45.239209 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:45.240461 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:45.241614 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:45.242238 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:45.242357 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:45.243088 (+ 731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:45.243092 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:45.243092 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:47:45.243294 (+ 202us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:45.243335 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:47:45.243348 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:45.243416 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:45.243511 (+ 95us) server_negotiation.cc:299] Negotiation successful
0806 22:47:45.243582 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":81,"threads_started":1}
I20260806 22:47:45.248921 19295 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:47:45.249415 19295 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:47:45.252274 19295 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b727570ae8664d178671d36f01ac7f38: Generated new TSK 0
I20260806 22:47:45.252463 19295 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:47:45.257920 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.65:0
--local_ip_for_outbound_sockets=127.6.40.65
--webserver_interface=127.6.40.65
--webserver_port=0
--tserver_master_addrs=127.6.40.126:40715
--builtin_ntp_servers=127.6.40.84:37609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260806 22:47:45.356765 19317 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:47:45.357162 19317 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:47:45.357239 19317 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:47:45.358966 19317 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:47:45.359027 19317 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:47:45.359071 19317 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.65
I20260806 22:47:45.360755 19317 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:37609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.40.65
--webserver_port=0
--tserver_master_addrs=127.6.40.126:40715
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19317
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.65
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:47:45.361074 19317 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:47:45.361397 19317 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:47:45.366789 19323 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:45.366851 19325 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:45.366819 19329 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:45.366976 19317 server_base.cc:1034] running on GCE node
I20260806 22:47:45.367246 19317 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:47:45.367540 19317 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:47:45.368734 19317 hybrid_clock.cc:648] HybridClock initialized: now 1786056465368713 us; error 33 us; skew 500 ppm
I20260806 22:47:45.370321 19317 webserver.cc:460] Webserver started at http://127.6.40.65:38025/ using document root <none> and password file <none>
I20260806 22:47:45.370618 19317 fs_manager.cc:359] Metadata directory not provided
I20260806 22:47:45.370684 19317 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:47:45.370823 19317 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:47:45.372296 19317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-0/data/instance:
uuid: "18c9b7e688294502849cadec016fef0d"
format_stamp: "Formatted at 2026-08-06 22:47:45 on dist-test-slave-zh2d"
server_key: "306850a0485c73851ff50f319bd11e45"
server_key_iv: "cb72a812cb532d444a41e4977f480c93"
server_key_version: "encryptionkey@0"
I20260806 22:47:45.372738 19317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-0/wal/instance:
uuid: "18c9b7e688294502849cadec016fef0d"
format_stamp: "Formatted at 2026-08-06 22:47:45 on dist-test-slave-zh2d"
server_key: "306850a0485c73851ff50f319bd11e45"
server_key_iv: "cb72a812cb532d444a41e4977f480c93"
server_key_version: "encryptionkey@0"
I20260806 22:47:45.374651 19317 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.001s
I20260806 22:47:45.375640 19335 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:45.375909 19317 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:47:45.376000 19317 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-0/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-0/wal
uuid: "18c9b7e688294502849cadec016fef0d"
format_stamp: "Formatted at 2026-08-06 22:47:45 on dist-test-slave-zh2d"
server_key: "306850a0485c73851ff50f319bd11e45"
server_key_iv: "cb72a812cb532d444a41e4977f480c93"
server_key_version: "encryptionkey@0"
I20260806 22:47:45.376076 19317 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:47:45.386720 19317 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:47:45.387120 19317 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:47:45.387282 19317 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:47:45.387544 19317 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:47:45.387928 19317 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:47:45.387981 19317 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:45.388011 19317 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:47:45.388047 19317 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:45.395191 19317 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.65:37355
I20260806 22:47:45.395247 19486 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.65:37355 every 8 connection(s)
I20260806 22:47:45.395591 19317 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
I20260806 22:47:45.401260 19312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:45.396797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.65:45837 (local address 127.6.40.126:40715)
0806 22:47:45.396866 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:45.396869 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:45.396926 (+ 57us) server_negotiation.cc:407] Connection header received
0806 22:47:45.397476 (+ 550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:45.397477 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:45.397491 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:45.397560 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:45.398226 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:45.398887 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:45.399875 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:45.400102 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:45.400162 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:45.401011 (+ 849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:45.401013 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:45.401013 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:47:45.401017 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:45.401045 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:47:45.401050 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:45.401095 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:45.401177 (+ 82us) server_negotiation.cc:299] Negotiation successful
0806 22:47:45.401199 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 22:47:45.401518 19490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:45.396527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:40715 (local address 127.6.40.65:45837)
0806 22:47:45.396744 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:45.396806 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:45.397357 (+ 551us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:45.397631 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:45.397635 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:45.397643 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:45.398129 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:45.398133 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:45.398982 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:45.398983 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:45.399774 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:45.399776 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:45.400183 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:45.400184 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:45.400295 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:45.400870 (+ 575us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:47:45.400871 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:45.400905 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:47:45.400933 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:47:45.401105 (+ 172us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:47:45.401110 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:47:45.401113 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:47:45.401164 (+ 51us) client_negotiation.cc:240] Negotiation successful
0806 22:47:45.401245 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":104,"threads_started":1}
I20260806 22:47:45.402098 19488 heartbeater.cc:347] Connected to a master server at 127.6.40.126:40715
I20260806 22:47:45.402204 19488 heartbeater.cc:464] Registering TS with master...
I20260806 22:47:45.402396 19488 heartbeater.cc:511] Master 127.6.40.126:40715 requested a full tablet report, sending...
I20260806 22:47:45.402945 19231 ts_manager.cc:194] Registered new tserver with Master: 18c9b7e688294502849cadec016fef0d (127.6.40.65:37355)
I20260806 22:47:45.403517 19231 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.40.65:45837
I20260806 22:47:45.404721 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 19317
I20260806 22:47:45.404801 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-0/wal/instance
I20260806 22:47:45.405030 6305 external_mini_cluster.cc:1428] Setting key 1a427a8a627659af35df251bb1fb346f
I20260806 22:47:45.405947 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.66:0
--local_ip_for_outbound_sockets=127.6.40.66
--webserver_interface=127.6.40.66
--webserver_port=0
--tserver_master_addrs=127.6.40.126:40715
--builtin_ntp_servers=127.6.40.84:37609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260806 22:47:45.492332 19491 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:47:45.492547 19491 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:47:45.492570 19491 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:47:45.493826 19491 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:47:45.493901 19491 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:47:45.493930 19491 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.66
I20260806 22:47:45.495329 19491 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:37609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.40.66
--webserver_port=0
--tserver_master_addrs=127.6.40.126:40715
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19491
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.66
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:47:45.495611 19491 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:47:45.495941 19491 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:47:45.501677 19497 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:45.501812 19491 server_base.cc:1034] running on GCE node
W20260806 22:47:45.501644 19499 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:45.501652 19502 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:45.502207 19491 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:47:45.502502 19491 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:47:45.503690 19491 hybrid_clock.cc:648] HybridClock initialized: now 1786056465503669 us; error 48 us; skew 500 ppm
I20260806 22:47:45.505147 19491 webserver.cc:460] Webserver started at http://127.6.40.66:45251/ using document root <none> and password file <none>
I20260806 22:47:45.505436 19491 fs_manager.cc:359] Metadata directory not provided
I20260806 22:47:45.505512 19491 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:47:45.505673 19491 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:47:45.506829 19491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-1/data/instance:
uuid: "ad8718e1dbea409db1d517cdec70a74a"
format_stamp: "Formatted at 2026-08-06 22:47:45 on dist-test-slave-zh2d"
server_key: "ea65935bab778dbd9f3622d8df927129"
server_key_iv: "485b4fe342985645fda1c1ec86b5a758"
server_key_version: "encryptionkey@0"
I20260806 22:47:45.507283 19491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-1/wal/instance:
uuid: "ad8718e1dbea409db1d517cdec70a74a"
format_stamp: "Formatted at 2026-08-06 22:47:45 on dist-test-slave-zh2d"
server_key: "ea65935bab778dbd9f3622d8df927129"
server_key_iv: "485b4fe342985645fda1c1ec86b5a758"
server_key_version: "encryptionkey@0"
I20260806 22:47:45.508884 19491 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:47:45.509851 19509 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:45.510070 19491 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:47:45.510181 19491 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-1/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-1/wal
uuid: "ad8718e1dbea409db1d517cdec70a74a"
format_stamp: "Formatted at 2026-08-06 22:47:45 on dist-test-slave-zh2d"
server_key: "ea65935bab778dbd9f3622d8df927129"
server_key_iv: "485b4fe342985645fda1c1ec86b5a758"
server_key_version: "encryptionkey@0"
I20260806 22:47:45.510268 19491 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:47:45.527984 19491 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:47:45.528337 19491 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:47:45.528502 19491 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:47:45.528746 19491 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:47:45.529165 19491 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:47:45.529223 19491 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:45.529259 19491 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:47:45.529286 19491 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:45.536077 19491 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.66:42533
I20260806 22:47:45.536129 19660 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.66:42533 every 8 connection(s)
I20260806 22:47:45.536463 19491 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
I20260806 22:47:45.542073 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 19491
I20260806 22:47:45.542182 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-1/wal/instance
I20260806 22:47:45.542307 19312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:45.537320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.66:39133 (local address 127.6.40.126:40715)
0806 22:47:45.537369 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:45.537371 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:45.537643 (+ 272us) server_negotiation.cc:407] Connection header received
0806 22:47:45.538346 (+ 703us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:45.538348 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:45.538362 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:45.538426 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:45.539135 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:45.539655 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:45.540690 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:45.540977 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:45.541052 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:45.541919 (+ 867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:45.541921 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:45.541921 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:47:45.541925 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:45.541940 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:47:45.541945 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:45.542000 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:45.542184 (+ 184us) server_negotiation.cc:299] Negotiation successful
0806 22:47:45.542221 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 22:47:45.542383 6305 external_mini_cluster.cc:1428] Setting key c04fb971815da797b51c08f2f5b85b03
I20260806 22:47:45.542567 19667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:45.537337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:40715 (local address 127.6.40.66:39133)
0806 22:47:45.537519 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:45.537555 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:45.538232 (+ 677us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:45.538502 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:45.538506 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:45.538523 (+ 17us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:45.539037 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:45.539041 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:45.539758 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:45.539760 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:45.540582 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:45.540584 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:45.541065 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:45.541066 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:45.541203 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:45.541746 (+ 543us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:47:45.541747 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:45.541776 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:47:45.541798 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:47:45.542023 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:47:45.542047 (+ 24us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:47:45.542051 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:47:45.542131 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 22:47:45.542255 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":84,"threads_started":1}
I20260806 22:47:45.543111 19661 heartbeater.cc:347] Connected to a master server at 127.6.40.126:40715
I20260806 22:47:45.543238 19661 heartbeater.cc:464] Registering TS with master...
I20260806 22:47:45.543435 19661 heartbeater.cc:511] Master 127.6.40.126:40715 requested a full tablet report, sending...
I20260806 22:47:45.543462 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.67:0
--local_ip_for_outbound_sockets=127.6.40.67
--webserver_interface=127.6.40.67
--webserver_port=0
--tserver_master_addrs=127.6.40.126:40715
--builtin_ntp_servers=127.6.40.84:37609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260806 22:47:45.543836 19231 ts_manager.cc:194] Registered new tserver with Master: ad8718e1dbea409db1d517cdec70a74a (127.6.40.66:42533)
I20260806 22:47:45.544163 19231 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.40.66:39133
W20260806 22:47:45.628644 19669 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:47:45.628901 19669 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:47:45.628957 19669 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:47:45.630199 19669 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:47:45.630254 19669 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:47:45.630293 19669 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.67
I20260806 22:47:45.631709 19669 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:37609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.40.67
--webserver_port=0
--tserver_master_addrs=127.6.40.126:40715
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19669
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.67
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:47:45.631966 19669 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:47:45.632331 19669 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:47:45.637737 19678 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:45.637828 19675 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:45.637835 19669 server_base.cc:1034] running on GCE node
W20260806 22:47:45.637984 19674 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:45.638204 19669 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:47:45.638545 19669 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:47:45.639737 19669 hybrid_clock.cc:648] HybridClock initialized: now 1786056465639725 us; error 40 us; skew 500 ppm
I20260806 22:47:45.641217 19669 webserver.cc:460] Webserver started at http://127.6.40.67:35105/ using document root <none> and password file <none>
I20260806 22:47:45.641479 19669 fs_manager.cc:359] Metadata directory not provided
I20260806 22:47:45.641551 19669 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:47:45.641706 19669 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:47:45.642786 19669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-2/data/instance:
uuid: "df6d4b1b63864d90a7853e2d282a2df8"
format_stamp: "Formatted at 2026-08-06 22:47:45 on dist-test-slave-zh2d"
server_key: "b34bcaae99c7d7c937b40fbf7d91c20e"
server_key_iv: "2e6b8516a5d56ebb5c6ff99d0739852a"
server_key_version: "encryptionkey@0"
I20260806 22:47:45.643153 19669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-2/wal/instance:
uuid: "df6d4b1b63864d90a7853e2d282a2df8"
format_stamp: "Formatted at 2026-08-06 22:47:45 on dist-test-slave-zh2d"
server_key: "b34bcaae99c7d7c937b40fbf7d91c20e"
server_key_iv: "2e6b8516a5d56ebb5c6ff99d0739852a"
server_key_version: "encryptionkey@0"
I20260806 22:47:45.644517 19669 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:47:45.645298 19685 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:45.645479 19669 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:45.645589 19669 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-2/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-2/wal
uuid: "df6d4b1b63864d90a7853e2d282a2df8"
format_stamp: "Formatted at 2026-08-06 22:47:45 on dist-test-slave-zh2d"
server_key: "b34bcaae99c7d7c937b40fbf7d91c20e"
server_key_iv: "2e6b8516a5d56ebb5c6ff99d0739852a"
server_key_version: "encryptionkey@0"
I20260806 22:47:45.645664 19669 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:47:45.664189 19669 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:47:45.664537 19669 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:47:45.664701 19669 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:47:45.664958 19669 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:47:45.665300 19669 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:47:45.665364 19669 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:45.665400 19669 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:47:45.665444 19669 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:45.673369 19669 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.67:33709
I20260806 22:47:45.673467 19849 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.67:33709 every 8 connection(s)
I20260806 22:47:45.673840 19669 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
I20260806 22:47:45.679751 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 19669
I20260806 22:47:45.679864 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-2/wal/instance
I20260806 22:47:45.680121 6305 external_mini_cluster.cc:1428] Setting key 9961e084b3edfde31d9e259557bbe824
I20260806 22:47:45.680227 19312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:45.674657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.67:42589 (local address 127.6.40.126:40715)
0806 22:47:45.674738 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:45.674742 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:45.675013 (+ 271us) server_negotiation.cc:407] Connection header received
0806 22:47:45.675704 (+ 691us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:45.675706 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:45.675720 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:45.675800 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:45.676759 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:45.677415 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:45.678549 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:45.678759 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:45.678836 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:45.679933 (+ 1097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:45.679935 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:45.679935 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:47:45.679942 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:45.679961 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:47:45.679966 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:45.680042 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:45.680106 (+ 64us) server_negotiation.cc:299] Negotiation successful
0806 22:47:45.680138 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 22:47:45.680523 19853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:45.674625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:40715 (local address 127.6.40.67:42589)
0806 22:47:45.674851 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:45.674898 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:45.675580 (+ 682us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:45.675950 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:45.675955 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:45.675967 (+ 12us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:45.676660 (+ 693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:45.676666 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:45.677515 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:45.677516 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:45.678440 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:45.678443 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:45.678848 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:45.678849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:45.678997 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:45.679749 (+ 752us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:47:45.679750 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:45.679779 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:47:45.679801 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:47:45.680045 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:47:45.680048 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:47:45.680051 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:47:45.680103 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 22:47:45.680220 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":105,"threads_started":1}
I20260806 22:47:45.680938 19850 heartbeater.cc:347] Connected to a master server at 127.6.40.126:40715
I20260806 22:47:45.681063 19850 heartbeater.cc:464] Registering TS with master...
I20260806 22:47:45.681267 19850 heartbeater.cc:511] Master 127.6.40.126:40715 requested a full tablet report, sending...
I20260806 22:47:45.681666 19229 ts_manager.cc:194] Registered new tserver with Master: df6d4b1b63864d90a7853e2d282a2df8 (127.6.40.67:33709)
I20260806 22:47:45.682057 19229 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.40.67:42589
I20260806 22:47:45.692121 6305 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 22:47:45.694219 19312 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:47:45.693609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55184 (local address 127.6.40.126:40715)
0806 22:47:45.693701 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:45.693704 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:45.693829 (+ 125us) server_negotiation.cc:407] Connection header received
0806 22:47:45.693924 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:45.693925 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:45.693938 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:45.693989 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:45.694142 (+ 153us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55184: BlockingRecv error: recv got EOF from 127.0.0.1:55184 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 22:47:45.694285 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 19317
I20260806 22:47:45.705318 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 19491
I20260806 22:47:45.715354 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 19669
I20260806 22:47:45.725224 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 19182
2026-08-06T22:47:45Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (654 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-06T22:47:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:47:45Z Disabled control of system clock
I20260806 22:47:45.756081 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056405724609-6305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056405724609-6305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056405724609-6305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.126:36579
--webserver_interface=127.6.40.126
--webserver_port=0
--builtin_ntp_servers=127.6.40.84:40759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.40.126:36579
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 22:47:45.840565 19865 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:47:45.840780 19865 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:47:45.840802 19865 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:47:45.841995 19865 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:47:45.842044 19865 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:47:45.842060 19865 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:47:45.842075 19865 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:47:45.842089 19865 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:47:45.843462 19865 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:40759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056405724609-6305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056405724609-6305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.40.126:36579
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.126:36579
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19865
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056405724609-6305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:47:45.843735 19865 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:47:45.844043 19865 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:47:45.848995 19871 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:45.849048 19872 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:45.849009 19874 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:45.849066 19865 server_base.cc:1034] running on GCE node
I20260806 22:47:45.849524 19865 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:47:45.849829 19865 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:47:45.851012 19865 hybrid_clock.cc:648] HybridClock initialized: now 1786056465850973 us; error 49 us; skew 500 ppm
I20260806 22:47:45.852428 19865 webserver.cc:460] Webserver started at http://127.6.40.126:41765/ using document root <none> and password file <none>
I20260806 22:47:45.852661 19865 fs_manager.cc:359] Metadata directory not provided
I20260806 22:47:45.852731 19865 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:47:45.852917 19865 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:47:45.854001 19865 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056405724609-6305-0/minicluster-data/master-0/data/instance:
uuid: "0b4e818744924cdabfd3fde9b1946c5a"
format_stamp: "Formatted at 2026-08-06 22:47:45 on dist-test-slave-zh2d"
server_key: "1d92e4a97d3a5d2cb3ba26d61b869897"
server_key_iv: "7cbf7325f931fa604db9bac017b04d14"
server_key_version: "encryptionkey@0"
I20260806 22:47:45.854385 19865 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056405724609-6305-0/minicluster-data/master-0/wal/instance:
uuid: "0b4e818744924cdabfd3fde9b1946c5a"
format_stamp: "Formatted at 2026-08-06 22:47:45 on dist-test-slave-zh2d"
server_key: "1d92e4a97d3a5d2cb3ba26d61b869897"
server_key_iv: "7cbf7325f931fa604db9bac017b04d14"
server_key_version: "encryptionkey@0"
I20260806 22:47:45.855698 19865 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:47:45.856459 19884 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:45.856635 19865 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:47:45.856745 19865 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056405724609-6305-0/minicluster-data/master-0/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056405724609-6305-0/minicluster-data/master-0/wal
uuid: "0b4e818744924cdabfd3fde9b1946c5a"
format_stamp: "Formatted at 2026-08-06 22:47:45 on dist-test-slave-zh2d"
server_key: "1d92e4a97d3a5d2cb3ba26d61b869897"
server_key_iv: "7cbf7325f931fa604db9bac017b04d14"
server_key_version: "encryptionkey@0"
I20260806 22:47:45.856819 19865 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056405724609-6305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056405724609-6305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056405724609-6305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:47:45.860118 19865 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:47:45.860394 19865 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:47:45.860548 19865 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:47:45.864346 19865 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.126:36579
I20260806 22:47:45.864424 19950 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.126:36579 every 8 connection(s)
I20260806 22:47:45.864725 19865 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
I20260806 22:47:45.865710 19951 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:47:45.867888 19951 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b4e818744924cdabfd3fde9b1946c5a: Bootstrap starting.
I20260806 22:47:45.868482 19951 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0b4e818744924cdabfd3fde9b1946c5a: Neither blocks nor log segments found. Creating new log.
I20260806 22:47:45.868788 19951 log.cc:824] T 00000000000000000000000000000000 P 0b4e818744924cdabfd3fde9b1946c5a: Log is configured to *not* fsync() on all Append() calls
I20260806 22:47:45.869508 19951 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b4e818744924cdabfd3fde9b1946c5a: No bootstrap required, opened a new log
I20260806 22:47:45.870450 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 19865
I20260806 22:47:45.870565 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056405724609-6305-0/minicluster-data/master-0/wal/instance
I20260806 22:47:45.870765 6305 external_mini_cluster.cc:1428] Setting key 37b8ce835710770699900cfc31acb2bd
I20260806 22:47:45.871639 19951 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0b4e818744924cdabfd3fde9b1946c5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b4e818744924cdabfd3fde9b1946c5a" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 36579 } }
I20260806 22:47:45.871791 19951 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0b4e818744924cdabfd3fde9b1946c5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:47:45.871835 19951 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0b4e818744924cdabfd3fde9b1946c5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b4e818744924cdabfd3fde9b1946c5a, State: Initialized, Role: FOLLOWER
I20260806 22:47:45.871977 19951 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0b4e818744924cdabfd3fde9b1946c5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b4e818744924cdabfd3fde9b1946c5a" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 36579 } }
I20260806 22:47:45.872082 19951 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0b4e818744924cdabfd3fde9b1946c5a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:47:45.872121 19951 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0b4e818744924cdabfd3fde9b1946c5a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:47:45.872164 19951 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0b4e818744924cdabfd3fde9b1946c5a [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:47:45.872900 19951 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0b4e818744924cdabfd3fde9b1946c5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b4e818744924cdabfd3fde9b1946c5a" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 36579 } }
I20260806 22:47:45.873009 19951 leader_election.cc:302] T 00000000000000000000000000000000 P 0b4e818744924cdabfd3fde9b1946c5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0b4e818744924cdabfd3fde9b1946c5a; no voters:
I20260806 22:47:45.873194 19951 leader_election.cc:288] T 00000000000000000000000000000000 P 0b4e818744924cdabfd3fde9b1946c5a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:47:45.873258 19959 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0b4e818744924cdabfd3fde9b1946c5a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:47:45.873401 19958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:45.871300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54688 (local address 127.6.40.126:36579)
0806 22:47:45.871553 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:45.871561 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:45.871597 (+ 36us) server_negotiation.cc:407] Connection header received
0806 22:47:45.871685 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:45.871690 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:45.871722 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:45.871859 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:45.872665 (+ 806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:45.872669 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:45.872670 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:47:45.872674 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:45.872720 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:47:45.872735 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:45.872816 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:45.872937 (+ 121us) server_negotiation.cc:299] Negotiation successful
0806 22:47:45.873035 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":96,"threads_started":1}
I20260806 22:47:45.873427 19959 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0b4e818744924cdabfd3fde9b1946c5a [term 1 LEADER]: Becoming Leader. State: Replica: 0b4e818744924cdabfd3fde9b1946c5a, State: Running, Role: LEADER
I20260806 22:47:45.873461 19951 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0b4e818744924cdabfd3fde9b1946c5a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:47:45.873536 19959 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0b4e818744924cdabfd3fde9b1946c5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b4e818744924cdabfd3fde9b1946c5a" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 36579 } }
I20260806 22:47:45.873842 19959 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b4e818744924cdabfd3fde9b1946c5a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0b4e818744924cdabfd3fde9b1946c5a. Latest consensus state: current_term: 1 leader_uuid: "0b4e818744924cdabfd3fde9b1946c5a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b4e818744924cdabfd3fde9b1946c5a" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 36579 } } }
I20260806 22:47:45.873931 19959 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b4e818744924cdabfd3fde9b1946c5a [sys.catalog]: This master's current role is: LEADER
I20260806 22:47:45.873852 19962 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b4e818744924cdabfd3fde9b1946c5a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0b4e818744924cdabfd3fde9b1946c5a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b4e818744924cdabfd3fde9b1946c5a" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 36579 } } }
I20260806 22:47:45.874166 19962 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b4e818744924cdabfd3fde9b1946c5a [sys.catalog]: This master's current role is: LEADER
I20260806 22:47:45.874312 19966 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:47:45.874976 19966 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:47:45.876626 19966 catalog_manager.cc:1347] Generated new cluster ID: 783112fd2ccb4dc2af903c955c1822f8
I20260806 22:47:45.876682 19966 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:47:45.883477 19966 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:47:45.883641 19966 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:47:45.891166 19966 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0b4e818744924cdabfd3fde9b1946c5a: Generated new TSK 0
I20260806 22:47:45.891373 19966 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:47:45.894387 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.65:0
--local_ip_for_outbound_sockets=127.6.40.65
--webserver_interface=127.6.40.65
--webserver_port=0
--tserver_master_addrs=127.6.40.126:36579
--builtin_ntp_servers=127.6.40.84:40759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 22:47:45.979267 19981 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:47:45.979509 19981 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:47:45.979553 19981 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:47:45.980757 19981 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:47:45.980798 19981 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:47:45.980824 19981 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.65
I20260806 22:47:45.982196 19981 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:40759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.40.65
--webserver_port=0
--tserver_master_addrs=127.6.40.126:36579
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19981
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.65
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:47:45.982465 19981 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:47:45.982792 19981 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:47:45.988051 19998 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:45.988094 20000 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:45.988095 19995 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:45.988708 19981 server_base.cc:1034] running on GCE node
I20260806 22:47:45.988986 19981 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:47:45.989264 19981 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:47:45.990430 19981 hybrid_clock.cc:648] HybridClock initialized: now 1786056465990402 us; error 44 us; skew 500 ppm
I20260806 22:47:45.991829 19981 webserver.cc:460] Webserver started at http://127.6.40.65:44589/ using document root <none> and password file <none>
I20260806 22:47:45.992123 19981 fs_manager.cc:359] Metadata directory not provided
I20260806 22:47:45.992169 19981 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:47:45.992312 19981 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:47:45.993484 19981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-0/data/instance:
uuid: "dd090c3c109547d1b2d7bb949ef69018"
format_stamp: "Formatted at 2026-08-06 22:47:45 on dist-test-slave-zh2d"
server_key: "a5eed44fee600b5ab79d1f45336f3229"
server_key_iv: "a9fddd65dc771de21e79a730b33beeba"
server_key_version: "encryptionkey@0"
I20260806 22:47:45.993854 19981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-0/wal/instance:
uuid: "dd090c3c109547d1b2d7bb949ef69018"
format_stamp: "Formatted at 2026-08-06 22:47:45 on dist-test-slave-zh2d"
server_key: "a5eed44fee600b5ab79d1f45336f3229"
server_key_iv: "a9fddd65dc771de21e79a730b33beeba"
server_key_version: "encryptionkey@0"
I20260806 22:47:45.995229 19981 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:47:45.996039 20008 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:45.996259 19981 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:47:45.996364 19981 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-0/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-0/wal
uuid: "dd090c3c109547d1b2d7bb949ef69018"
format_stamp: "Formatted at 2026-08-06 22:47:45 on dist-test-slave-zh2d"
server_key: "a5eed44fee600b5ab79d1f45336f3229"
server_key_iv: "a9fddd65dc771de21e79a730b33beeba"
server_key_version: "encryptionkey@0"
I20260806 22:47:45.996454 19981 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:47:45.999864 19981 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:47:46.000186 19981 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:47:46.000375 19981 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:47:46.000607 19981 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:47:46.001008 19981 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:47:46.001060 19981 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:46.001101 19981 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:47:46.001128 19981 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:46.007386 19981 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.65:43615
I20260806 22:47:46.007493 20163 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.65:43615 every 8 connection(s)
I20260806 22:47:46.007817 19981 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
I20260806 22:47:46.010054 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 19981
I20260806 22:47:46.010159 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-0/wal/instance
I20260806 22:47:46.010392 6305 external_mini_cluster.cc:1428] Setting key 8fc4fe65c44a21709db7356f19451803
I20260806 22:47:46.011276 19958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:46.008574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.65:52171 (local address 127.6.40.126:36579)
0806 22:47:46.008661 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:46.008664 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:46.008972 (+ 308us) server_negotiation.cc:407] Connection header received
0806 22:47:46.009468 (+ 496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:46.009469 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:46.009475 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:46.009533 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:46.010951 (+ 1418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:46.010952 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:46.010952 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:47:46.010956 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:46.010972 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:47:46.010976 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:46.011026 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:46.011187 (+ 161us) server_negotiation.cc:299] Negotiation successful
0806 22:47:46.011203 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 22:47:46.011479 20166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:46.008551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:36579 (local address 127.6.40.65:52171)
0806 22:47:46.008788 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:46.008823 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:46.009375 (+ 552us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:46.009582 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:46.009585 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:46.009594 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:46.010791 (+ 1197us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:47:46.010792 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:46.010830 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:47:46.010866 (+ 36us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:47:46.011046 (+ 180us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:47:46.011053 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:47:46.011056 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:47:46.011173 (+ 117us) client_negotiation.cc:240] Negotiation successful
0806 22:47:46.011240 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":135,"threads_started":1}
I20260806 22:47:46.011524 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.66:0
--local_ip_for_outbound_sockets=127.6.40.66
--webserver_interface=127.6.40.66
--webserver_port=0
--tserver_master_addrs=127.6.40.126:36579
--builtin_ntp_servers=127.6.40.84:40759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260806 22:47:46.011991 20164 heartbeater.cc:347] Connected to a master server at 127.6.40.126:36579
I20260806 22:47:46.012084 20164 heartbeater.cc:464] Registering TS with master...
I20260806 22:47:46.012239 20164 heartbeater.cc:511] Master 127.6.40.126:36579 requested a full tablet report, sending...
I20260806 22:47:46.012777 19909 ts_manager.cc:194] Registered new tserver with Master: dd090c3c109547d1b2d7bb949ef69018 (127.6.40.65:43615)
W20260806 22:47:46.096599 20168 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:47:46.096876 20168 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:47:46.096921 20168 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:47:46.098142 20168 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:47:46.098213 20168 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:47:46.098258 20168 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.66
I20260806 22:47:46.099866 20168 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:40759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.40.66
--webserver_port=0
--tserver_master_addrs=127.6.40.126:36579
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20168
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.66
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:47:46.100129 20168 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:47:46.100481 20168 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:47:46.105625 20173 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:46.105757 20178 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:46.105798 20174 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:46.106076 20168 server_base.cc:1034] running on GCE node
I20260806 22:47:46.106279 20168 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:47:46.106547 20168 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:47:46.107715 20168 hybrid_clock.cc:648] HybridClock initialized: now 1786056466107702 us; error 29 us; skew 500 ppm
I20260806 22:47:46.109050 20168 webserver.cc:460] Webserver started at http://127.6.40.66:33185/ using document root <none> and password file <none>
I20260806 22:47:46.109299 20168 fs_manager.cc:359] Metadata directory not provided
I20260806 22:47:46.109369 20168 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:47:46.109522 20168 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:47:46.110596 20168 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-1/data/instance:
uuid: "1df8843a228940caa6076321e4a7737c"
format_stamp: "Formatted at 2026-08-06 22:47:46 on dist-test-slave-zh2d"
server_key: "2d4a6ad18dd88c1b1aaa842849199ffc"
server_key_iv: "fcfa10152dd87b541c2d0af7b9a2e0f8"
server_key_version: "encryptionkey@0"
I20260806 22:47:46.111063 20168 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-1/wal/instance:
uuid: "1df8843a228940caa6076321e4a7737c"
format_stamp: "Formatted at 2026-08-06 22:47:46 on dist-test-slave-zh2d"
server_key: "2d4a6ad18dd88c1b1aaa842849199ffc"
server_key_iv: "fcfa10152dd87b541c2d0af7b9a2e0f8"
server_key_version: "encryptionkey@0"
I20260806 22:47:46.112558 20168 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:47:46.113445 20185 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:46.113662 20168 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 22:47:46.113766 20168 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-1/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-1/wal
uuid: "1df8843a228940caa6076321e4a7737c"
format_stamp: "Formatted at 2026-08-06 22:47:46 on dist-test-slave-zh2d"
server_key: "2d4a6ad18dd88c1b1aaa842849199ffc"
server_key_iv: "fcfa10152dd87b541c2d0af7b9a2e0f8"
server_key_version: "encryptionkey@0"
I20260806 22:47:46.113848 20168 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:47:46.117281 20168 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:47:46.117574 20168 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:47:46.117719 20168 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:47:46.117941 20168 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:47:46.118242 20168 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:47:46.118299 20168 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:46.118337 20168 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:47:46.118366 20168 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:46.124439 20168 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.66:44353
I20260806 22:47:46.124493 20347 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.66:44353 every 8 connection(s)
I20260806 22:47:46.124806 20168 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
I20260806 22:47:46.127413 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 20168
I20260806 22:47:46.127533 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-1/wal/instance
I20260806 22:47:46.127736 6305 external_mini_cluster.cc:1428] Setting key 076040fba7f2a6313080ae026333b5d6
I20260806 22:47:46.128726 19958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:46.125665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.66:43907 (local address 127.6.40.126:36579)
0806 22:47:46.125731 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:46.125733 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:46.126076 (+ 343us) server_negotiation.cc:407] Connection header received
0806 22:47:46.126835 (+ 759us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:46.126836 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:46.126843 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:46.126905 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:46.128361 (+ 1456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:46.128362 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:46.128363 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:47:46.128366 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:46.128431 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:47:46.128437 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:46.128522 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:46.128634 (+ 112us) server_negotiation.cc:299] Negotiation successful
0806 22:47:46.128650 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 22:47:46.128960 20350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:46.125679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:36579 (local address 127.6.40.66:43907)
0806 22:47:46.125917 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:46.125958 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:46.126704 (+ 746us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:46.126957 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:46.126960 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:46.126968 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:46.128211 (+ 1243us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:47:46.128212 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:46.128247 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:47:46.128279 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:47:46.128541 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:47:46.128548 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:47:46.128551 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:47:46.128635 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 22:47:46.128701 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":108,"threads_started":1}
I20260806 22:47:46.129022 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.67:0
--local_ip_for_outbound_sockets=127.6.40.67
--webserver_interface=127.6.40.67
--webserver_port=0
--tserver_master_addrs=127.6.40.126:36579
--builtin_ntp_servers=127.6.40.84:40759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260806 22:47:46.129534 20348 heartbeater.cc:347] Connected to a master server at 127.6.40.126:36579
I20260806 22:47:46.129622 20348 heartbeater.cc:464] Registering TS with master...
I20260806 22:47:46.129770 20348 heartbeater.cc:511] Master 127.6.40.126:36579 requested a full tablet report, sending...
I20260806 22:47:46.130136 19909 ts_manager.cc:194] Registered new tserver with Master: 1df8843a228940caa6076321e4a7737c (127.6.40.66:44353)
W20260806 22:47:46.220242 20351 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:47:46.220479 20351 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:47:46.220520 20351 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:47:46.221763 20351 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:47:46.221828 20351 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:47:46.221871 20351 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.67
I20260806 22:47:46.223217 20351 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:40759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.40.67
--webserver_port=0
--tserver_master_addrs=127.6.40.126:36579
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20351
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.67
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:47:46.223482 20351 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:47:46.223796 20351 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:47:46.230755 20359 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:46.230932 20362 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:46.230820 20360 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:46.230935 20351 server_base.cc:1034] running on GCE node
I20260806 22:47:46.231479 20351 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:47:46.231760 20351 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:47:46.232930 20351 hybrid_clock.cc:648] HybridClock initialized: now 1786056466232914 us; error 34 us; skew 500 ppm
I20260806 22:47:46.234709 20351 webserver.cc:460] Webserver started at http://127.6.40.67:43293/ using document root <none> and password file <none>
I20260806 22:47:46.234954 20351 fs_manager.cc:359] Metadata directory not provided
I20260806 22:47:46.235023 20351 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:47:46.235164 20351 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:47:46.236267 20351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-2/data/instance:
uuid: "446a1d4ff1ae4a49b775fef193d3f1e1"
format_stamp: "Formatted at 2026-08-06 22:47:46 on dist-test-slave-zh2d"
server_key: "053778b0301f48925225f37d83779f60"
server_key_iv: "d4760e3ae40d424abce9f457923b6e57"
server_key_version: "encryptionkey@0"
I20260806 22:47:46.236667 20351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-2/wal/instance:
uuid: "446a1d4ff1ae4a49b775fef193d3f1e1"
format_stamp: "Formatted at 2026-08-06 22:47:46 on dist-test-slave-zh2d"
server_key: "053778b0301f48925225f37d83779f60"
server_key_iv: "d4760e3ae40d424abce9f457923b6e57"
server_key_version: "encryptionkey@0"
I20260806 22:47:46.238181 20351 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:47:46.239045 20368 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:46.239235 20351 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:47:46.239378 20351 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-2/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-2/wal
uuid: "446a1d4ff1ae4a49b775fef193d3f1e1"
format_stamp: "Formatted at 2026-08-06 22:47:46 on dist-test-slave-zh2d"
server_key: "053778b0301f48925225f37d83779f60"
server_key_iv: "d4760e3ae40d424abce9f457923b6e57"
server_key_version: "encryptionkey@0"
I20260806 22:47:46.239459 20351 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:47:46.242838 20351 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:47:46.243112 20351 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:47:46.243265 20351 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:47:46.243494 20351 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:47:46.243795 20351 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:47:46.243848 20351 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:46.243891 20351 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:47:46.243917 20351 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:46.250638 20351 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.67:34953
I20260806 22:47:46.250721 20533 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.67:34953 every 8 connection(s)
I20260806 22:47:46.251001 20351 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
I20260806 22:47:46.254997 19958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:46.251795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.67:43265 (local address 127.6.40.126:36579)
0806 22:47:46.251877 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:46.251880 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:46.252189 (+ 309us) server_negotiation.cc:407] Connection header received
0806 22:47:46.252997 (+ 808us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:46.252998 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:46.253004 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:46.253053 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:46.254630 (+ 1577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:46.254631 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:46.254632 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:47:46.254636 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:46.254650 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:47:46.254654 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:46.254715 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:46.254880 (+ 165us) server_negotiation.cc:299] Negotiation successful
0806 22:47:46.254907 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 22:47:46.255036 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 20351
I20260806 22:47:46.255128 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056405724609-6305-0/minicluster-data/ts-2/wal/instance
I20260806 22:47:46.255324 6305 external_mini_cluster.cc:1428] Setting key 2f1d529a1a3562b8780fd957a95db54a
I20260806 22:47:46.255475 20536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:46.251771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:36579 (local address 127.6.40.67:43265)
0806 22:47:46.252019 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:46.252073 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:46.252862 (+ 789us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:46.253152 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:46.253158 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:46.253170 (+ 12us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:46.254409 (+ 1239us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:47:46.254410 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:46.254453 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:47:46.254492 (+ 39us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:47:46.254733 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:47:46.254739 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:47:46.254742 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:47:46.254835 (+ 93us) client_negotiation.cc:240] Negotiation successful
0806 22:47:46.254913 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":119,"threads_started":1}
I20260806 22:47:46.255878 20534 heartbeater.cc:347] Connected to a master server at 127.6.40.126:36579
I20260806 22:47:46.256006 20534 heartbeater.cc:464] Registering TS with master...
I20260806 22:47:46.256163 20534 heartbeater.cc:511] Master 127.6.40.126:36579 requested a full tablet report, sending...
I20260806 22:47:46.256582 19908 ts_manager.cc:194] Registered new tserver with Master: 446a1d4ff1ae4a49b775fef193d3f1e1 (127.6.40.67:34953)
I20260806 22:47:46.267328 6305 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 22:47:46.269835 19958 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:47:46.269197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54690 (local address 127.6.40.126:36579)
0806 22:47:46.269266 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:46.269269 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:46.269450 (+ 181us) server_negotiation.cc:407] Connection header received
0806 22:47:46.269520 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:46.269520 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:46.269527 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:46.269575 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:46.269761 (+ 186us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54690: BlockingRecv error: recv got EOF from 127.0.0.1:54690 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 22:47:46.269863 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 19981
I20260806 22:47:46.296257 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 20168
I20260806 22:47:46.313238 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 20351
I20260806 22:47:46.320830 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 19865
2026-08-06T22:47:46Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (596 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:47:46 dist-test-slave-zh2d krb5kdc[20547](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:47:46 dist-test-slave-zh2d krb5kdc[20547](info): set up 2 sockets
Aug 06 22:47:46 dist-test-slave-zh2d krb5kdc[20547](info): commencing operation
krb5kdc: starting...
W20260806 22:47:47.545840 6305 mini_kdc.cc:121] Time spent starting KDC: real 1.201s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/krb5kdc/test-user.keytab.
Aug 06 22:47:47 dist-test-slave-zh2d krb5kdc[20547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056467, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:47:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:47:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.40.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.40.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.40.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab.
Entry for principal kudu/127.6.40.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab.
I20260806 22:47:47.669795 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.126:40225
--webserver_interface=127.6.40.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab
--principal=kudu/127.6.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:40519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.40.126:40225
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:47:47.756583 20566 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:47:47.756801 20566 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:47:47.756826 20566 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:47:47.758029 20566 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:47:47.758078 20566 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:47:47.758095 20566 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:47:47.758174 20566 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:47:47.758203 20566 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:47:47.759748 20566 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:40519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.40.126:40225
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab
--principal=kudu/127.6.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.126:40225
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20566
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:47:47.760018 20566 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:47:47.760264 20566 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 22:47:47.765635 20566 server_base.cc:1034] running on GCE node
W20260806 22:47:47.765602 20576 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:47.765612 20574 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:47.765872 20573 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:47.766202 20566 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:47:47.766453 20566 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:47:47.767622 20566 hybrid_clock.cc:648] HybridClock initialized: now 1786056467767616 us; error 37 us; skew 500 ppm
Aug 06 22:47:47 dist-test-slave-zh2d krb5kdc[20547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056467, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:47:47.770212 20566 init.cc:377] Logged in from keytab as kudu/127.6.40.126@KRBTEST.COM (short username kudu)
I20260806 22:47:47.770691 20566 webserver.cc:460] Webserver started at http://127.6.40.126:39923/ using document root <none> and password file <none>
I20260806 22:47:47.770923 20566 fs_manager.cc:359] Metadata directory not provided
I20260806 22:47:47.770995 20566 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:47:47.771152 20566 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:47:47.772241 20566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/minicluster-data/master-0/data/instance:
uuid: "036aefa83777434fb51d359c1e235b0b"
format_stamp: "Formatted at 2026-08-06 22:47:47 on dist-test-slave-zh2d"
server_key: "e5e0fcf952981482c95d64e16e8fb934"
server_key_iv: "b6dee5820f84ed6f47cc16b2524b59db"
server_key_version: "encryptionkey@0"
I20260806 22:47:47.772608 20566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/minicluster-data/master-0/wal/instance:
uuid: "036aefa83777434fb51d359c1e235b0b"
format_stamp: "Formatted at 2026-08-06 22:47:47 on dist-test-slave-zh2d"
server_key: "e5e0fcf952981482c95d64e16e8fb934"
server_key_iv: "b6dee5820f84ed6f47cc16b2524b59db"
server_key_version: "encryptionkey@0"
I20260806 22:47:47.774088 20566 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:47:47.774859 20587 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:47.775087 20566 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:47:47.775192 20566 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/minicluster-data/master-0/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/minicluster-data/master-0/wal
uuid: "036aefa83777434fb51d359c1e235b0b"
format_stamp: "Formatted at 2026-08-06 22:47:47 on dist-test-slave-zh2d"
server_key: "e5e0fcf952981482c95d64e16e8fb934"
server_key_iv: "b6dee5820f84ed6f47cc16b2524b59db"
server_key_version: "encryptionkey@0"
I20260806 22:47:47.775279 20566 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:47:47.802413 20566 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:47:47.804456 20566 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:47:47.804637 20566 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:47:47.809078 20566 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.126:40225
I20260806 22:47:47.809119 20666 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.126:40225 every 8 connection(s)
I20260806 22:47:47.809504 20566 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
I20260806 22:47:47.810520 20668 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:47:47.812871 20668 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 036aefa83777434fb51d359c1e235b0b: Bootstrap starting.
I20260806 22:47:47.813488 20668 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 036aefa83777434fb51d359c1e235b0b: Neither blocks nor log segments found. Creating new log.
I20260806 22:47:47.813735 20668 log.cc:824] T 00000000000000000000000000000000 P 036aefa83777434fb51d359c1e235b0b: Log is configured to *not* fsync() on all Append() calls
I20260806 22:47:47.814437 20668 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 036aefa83777434fb51d359c1e235b0b: No bootstrap required, opened a new log
I20260806 22:47:47.816080 20668 raft_consensus.cc:348] T 00000000000000000000000000000000 P 036aefa83777434fb51d359c1e235b0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "036aefa83777434fb51d359c1e235b0b" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 40225 } }
I20260806 22:47:47.816224 20668 raft_consensus.cc:374] T 00000000000000000000000000000000 P 036aefa83777434fb51d359c1e235b0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:47:47.816246 20668 raft_consensus.cc:729] T 00000000000000000000000000000000 P 036aefa83777434fb51d359c1e235b0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 036aefa83777434fb51d359c1e235b0b, State: Initialized, Role: FOLLOWER
I20260806 22:47:47.816424 20668 consensus_queue.cc:260] T 00000000000000000000000000000000 P 036aefa83777434fb51d359c1e235b0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "036aefa83777434fb51d359c1e235b0b" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 40225 } }
I20260806 22:47:47.816505 20668 raft_consensus.cc:388] T 00000000000000000000000000000000 P 036aefa83777434fb51d359c1e235b0b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:47:47.816540 20668 raft_consensus.cc:482] T 00000000000000000000000000000000 P 036aefa83777434fb51d359c1e235b0b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:47:47.816522 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 20566
I20260806 22:47:47.816592 20668 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 036aefa83777434fb51d359c1e235b0b [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:47:47.816636 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/minicluster-data/master-0/wal/instance
I20260806 22:47:47.816814 6305 external_mini_cluster.cc:1428] Setting key cfcad6d378b23ea8e3774ecb44a5931e
I20260806 22:47:47.817754 20668 raft_consensus.cc:504] T 00000000000000000000000000000000 P 036aefa83777434fb51d359c1e235b0b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "036aefa83777434fb51d359c1e235b0b" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 40225 } }
I20260806 22:47:47.817848 20668 leader_election.cc:302] T 00000000000000000000000000000000 P 036aefa83777434fb51d359c1e235b0b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 036aefa83777434fb51d359c1e235b0b; no voters:
I20260806 22:47:47.818040 20668 leader_election.cc:288] T 00000000000000000000000000000000 P 036aefa83777434fb51d359c1e235b0b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:47:47.818135 20678 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 036aefa83777434fb51d359c1e235b0b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:47:47.818315 20668 sys_catalog.cc:564] T 00000000000000000000000000000000 P 036aefa83777434fb51d359c1e235b0b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:47:47.818315 20678 raft_consensus.cc:686] T 00000000000000000000000000000000 P 036aefa83777434fb51d359c1e235b0b [term 1 LEADER]: Becoming Leader. State: Replica: 036aefa83777434fb51d359c1e235b0b, State: Running, Role: LEADER
I20260806 22:47:47.818467 20678 consensus_queue.cc:237] T 00000000000000000000000000000000 P 036aefa83777434fb51d359c1e235b0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "036aefa83777434fb51d359c1e235b0b" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 40225 } }
I20260806 22:47:47.818835 20678 sys_catalog.cc:455] T 00000000000000000000000000000000 P 036aefa83777434fb51d359c1e235b0b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 036aefa83777434fb51d359c1e235b0b. Latest consensus state: current_term: 1 leader_uuid: "036aefa83777434fb51d359c1e235b0b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "036aefa83777434fb51d359c1e235b0b" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 40225 } } }
I20260806 22:47:47.818938 20678 sys_catalog.cc:458] T 00000000000000000000000000000000 P 036aefa83777434fb51d359c1e235b0b [sys.catalog]: This master's current role is: LEADER
I20260806 22:47:47.819126 20679 sys_catalog.cc:455] T 00000000000000000000000000000000 P 036aefa83777434fb51d359c1e235b0b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "036aefa83777434fb51d359c1e235b0b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "036aefa83777434fb51d359c1e235b0b" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 40225 } } }
I20260806 22:47:47.819217 20679 sys_catalog.cc:458] T 00000000000000000000000000000000 P 036aefa83777434fb51d359c1e235b0b [sys.catalog]: This master's current role is: LEADER
I20260806 22:47:47.819339 20691 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:47:47.819922 20691 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:47:47.821868 20691 catalog_manager.cc:1347] Generated new cluster ID: e59fd1cc4f3943c5a175737b49314600
I20260806 22:47:47.821926 20691 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 22:47:47 dist-test-slave-zh2d krb5kdc[20547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056467, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
I20260806 22:47:47.828336 20675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:47.817406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49152 (local address 127.6.40.126:40225)
0806 22:47:47.817606 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:47.817622 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:47.817650 (+ 28us) server_negotiation.cc:407] Connection header received
0806 22:47:47.817745 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:47.817751 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:47.817795 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:47.817976 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:47.818846 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:47.820460 (+ 1614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:47.821533 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:47.822112 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:47.822243 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:47.824731 (+ 2488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:47.824736 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:47.824736 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:47.824751 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:47.826660 (+ 1909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:47.827129 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:47.827134 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:47.827139 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:47.827240 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:47.827470 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:47.827471 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:47.827472 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:47.827722 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:47.827910 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:47.828014 (+ 104us) server_negotiation.cc:299] Negotiation successful
0806 22:47:47.828163 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":135,"threads_started":1}
I20260806 22:47:47.833760 20691 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:47:47.834455 20691 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:47:47.841136 20691 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 036aefa83777434fb51d359c1e235b0b: Generated new TSK 0
I20260806 22:47:47.841315 20691 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.40.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.40.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.40.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.65.keytab.
Entry for principal kudu/127.6.40.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.65.keytab.
I20260806 22:47:47.880779 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.65:0
--local_ip_for_outbound_sockets=127.6.40.65
--webserver_interface=127.6.40.65
--webserver_port=0
--tserver_master_addrs=127.6.40.126:40225
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.65.keytab
--principal=kudu/127.6.40.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:40519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:47:47.984470 20710 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:47:47.984688 20710 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:47:47.984715 20710 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:47:47.985999 20710 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:47:47.986049 20710 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:47:47.986083 20710 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.65
I20260806 22:47:47.987668 20710 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:40519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.65.keytab
--principal=kudu/127.6.40.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.40.65
--webserver_port=0
--tserver_master_addrs=127.6.40.126:40225
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20710
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.65
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:47:47.987939 20710 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:47:47.988205 20710 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:47:47.993883 20719 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:47.993942 20716 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:47.994065 20710 server_base.cc:1034] running on GCE node
W20260806 22:47:47.993883 20717 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:47.994421 20710 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:47:47.994720 20710 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:47:47.995887 20710 hybrid_clock.cc:648] HybridClock initialized: now 1786056467995866 us; error 39 us; skew 500 ppm
Aug 06 22:47:47 dist-test-slave-zh2d krb5kdc[20547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056467, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:47:48.004237 20710 init.cc:377] Logged in from keytab as kudu/127.6.40.65@KRBTEST.COM (short username kudu)
I20260806 22:47:48.004783 20710 webserver.cc:460] Webserver started at http://127.6.40.65:37709/ using document root <none> and password file <none>
I20260806 22:47:48.005043 20710 fs_manager.cc:359] Metadata directory not provided
I20260806 22:47:48.005091 20710 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:47:48.005187 20710 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:47:48.006276 20710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/minicluster-data/ts-0/data/instance:
uuid: "d1083c944f814031a6268a9fadbc7d78"
format_stamp: "Formatted at 2026-08-06 22:47:48 on dist-test-slave-zh2d"
server_key: "8250f8e7a9f0a97faeb5ca83835bbce1"
server_key_iv: "0c2efe6501d84bf0469013c915c197cd"
server_key_version: "encryptionkey@0"
I20260806 22:47:48.006603 20710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/minicluster-data/ts-0/wal/instance:
uuid: "d1083c944f814031a6268a9fadbc7d78"
format_stamp: "Formatted at 2026-08-06 22:47:48 on dist-test-slave-zh2d"
server_key: "8250f8e7a9f0a97faeb5ca83835bbce1"
server_key_iv: "0c2efe6501d84bf0469013c915c197cd"
server_key_version: "encryptionkey@0"
I20260806 22:47:48.007944 20710 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:47:48.008738 20728 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:48.009009 20710 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:47:48.009095 20710 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/minicluster-data/ts-0/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/minicluster-data/ts-0/wal
uuid: "d1083c944f814031a6268a9fadbc7d78"
format_stamp: "Formatted at 2026-08-06 22:47:48 on dist-test-slave-zh2d"
server_key: "8250f8e7a9f0a97faeb5ca83835bbce1"
server_key_iv: "0c2efe6501d84bf0469013c915c197cd"
server_key_version: "encryptionkey@0"
I20260806 22:47:48.009150 20710 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:47:48.062497 20710 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:47:48.064988 20710 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:47:48.065110 20710 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:47:48.065325 20710 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:47:48.065687 20710 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:47:48.065725 20710 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:48.065781 20710 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:47:48.065807 20710 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:48.071825 20710 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.65:46423
I20260806 22:47:48.071877 20892 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.65:46423 every 8 connection(s)
I20260806 22:47:48.072199 20710 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
I20260806 22:47:48.077845 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 20710
I20260806 22:47:48.077965 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/minicluster-data/ts-0/wal/instance
I20260806 22:47:48.078195 6305 external_mini_cluster.cc:1428] Setting key a87ad2cd83da8355849fe0a9a97196cb
Aug 06 22:47:48 dist-test-slave-zh2d krb5kdc[20547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056467, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.65@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
I20260806 22:47:48.082119 20675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:48.072941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.65:60785 (local address 127.6.40.126:40225)
0806 22:47:48.072990 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:48.072992 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:48.073219 (+ 227us) server_negotiation.cc:407] Connection header received
0806 22:47:48.073738 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:48.073739 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:48.073755 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:48.073808 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:48.074725 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:48.075224 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:48.076196 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:48.076428 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:48.076487 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:48.078837 (+ 2350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:48.078839 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:48.078839 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:48.078847 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:48.080576 (+ 1729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:48.081079 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:48.081085 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:48.081086 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:48.081142 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:48.081452 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:48.081453 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:48.081453 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:48.081560 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:48.081634 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:48.081924 (+ 290us) server_negotiation.cc:299] Negotiation successful
0806 22:47:48.082024 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 22:47:48.082322 20896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:48.072908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:40225 (local address 127.6.40.65:60785)
0806 22:47:48.073105 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:48.073144 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:48.073647 (+ 503us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:48.073862 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:48.073865 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:48.074448 (+ 583us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:48.074646 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:48.074651 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:48.075309 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:48.075310 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:48.076108 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:48.076110 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:48.076509 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:48.076509 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:48.076620 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:48.077124 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:48.077132 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:48.078726 (+ 1594us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:47:48.080664 (+ 1938us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:48.080671 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:48.080672 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:48.080987 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:48.081208 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:48.081209 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:48.081210 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:48.081336 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:48.081637 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:47:48.081645 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:47:48.081785 (+ 140us) client_negotiation.cc:769] Sending connection context
0806 22:47:48.081904 (+ 119us) client_negotiation.cc:240] Negotiation successful
0806 22:47:48.082069 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":98,"threads_started":1}
I20260806 22:47:48.082795 20894 heartbeater.cc:347] Connected to a master server at 127.6.40.126:40225
I20260806 22:47:48.082913 20894 heartbeater.cc:464] Registering TS with master...
I20260806 22:47:48.083098 20894 heartbeater.cc:511] Master 127.6.40.126:40225 requested a full tablet report, sending...
I20260806 22:47:48.083630 20614 ts_manager.cc:194] Registered new tserver with Master: d1083c944f814031a6268a9fadbc7d78 (127.6.40.65:46423)
I20260806 22:47:48.084645 20614 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.40.65@KRBTEST.COM'} at 127.6.40.65:60785
WARNING: no policy specified for kudu/127.6.40.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.40.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.40.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.66.keytab.
Entry for principal kudu/127.6.40.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.66.keytab.
I20260806 22:47:48.108791 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.66:0
--local_ip_for_outbound_sockets=127.6.40.66
--webserver_interface=127.6.40.66
--webserver_port=0
--tserver_master_addrs=127.6.40.126:40225
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.66.keytab
--principal=kudu/127.6.40.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:40519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:47:48.195407 20900 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:47:48.195614 20900 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:47:48.195638 20900 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:47:48.196878 20900 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:47:48.196928 20900 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:47:48.196955 20900 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.66
I20260806 22:47:48.198434 20900 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:40519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.66.keytab
--principal=kudu/127.6.40.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.40.66
--webserver_port=0
--tserver_master_addrs=127.6.40.126:40225
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20900
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.66
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:47:48.198678 20900 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:47:48.198916 20900 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:47:48.204274 20908 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:48.204203 20907 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:48.204202 20910 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:48.204596 20900 server_base.cc:1034] running on GCE node
I20260806 22:47:48.204921 20900 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:47:48.205180 20900 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:47:48.206343 20900 hybrid_clock.cc:648] HybridClock initialized: now 1786056468206331 us; error 30 us; skew 500 ppm
Aug 06 22:47:48 dist-test-slave-zh2d krb5kdc[20547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056468, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:47:48.209134 20900 init.cc:377] Logged in from keytab as kudu/127.6.40.66@KRBTEST.COM (short username kudu)
I20260806 22:47:48.209597 20900 webserver.cc:460] Webserver started at http://127.6.40.66:45615/ using document root <none> and password file <none>
I20260806 22:47:48.209781 20900 fs_manager.cc:359] Metadata directory not provided
I20260806 22:47:48.209822 20900 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:47:48.209923 20900 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:47:48.210934 20900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/minicluster-data/ts-1/data/instance:
uuid: "e26d7748ce7747148c2c3ce204b73166"
format_stamp: "Formatted at 2026-08-06 22:47:48 on dist-test-slave-zh2d"
server_key: "dd5edc651cf811dccc0e317e0349d198"
server_key_iv: "6c02069405a239fbad0ec89dfcb31244"
server_key_version: "encryptionkey@0"
I20260806 22:47:48.211246 20900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/minicluster-data/ts-1/wal/instance:
uuid: "e26d7748ce7747148c2c3ce204b73166"
format_stamp: "Formatted at 2026-08-06 22:47:48 on dist-test-slave-zh2d"
server_key: "dd5edc651cf811dccc0e317e0349d198"
server_key_iv: "6c02069405a239fbad0ec89dfcb31244"
server_key_version: "encryptionkey@0"
I20260806 22:47:48.212689 20900 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 22:47:48.213517 20920 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:48.213707 20900 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:47:48.213792 20900 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/minicluster-data/ts-1/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/minicluster-data/ts-1/wal
uuid: "e26d7748ce7747148c2c3ce204b73166"
format_stamp: "Formatted at 2026-08-06 22:47:48 on dist-test-slave-zh2d"
server_key: "dd5edc651cf811dccc0e317e0349d198"
server_key_iv: "6c02069405a239fbad0ec89dfcb31244"
server_key_version: "encryptionkey@0"
I20260806 22:47:48.213848 20900 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:47:48.237823 20900 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:47:48.239758 20900 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:47:48.239877 20900 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:47:48.240092 20900 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:47:48.240417 20900 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:47:48.240450 20900 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:48.240496 20900 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:47:48.240523 20900 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:48.246635 20900 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.66:37189
I20260806 22:47:48.246727 21073 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.66:37189 every 8 connection(s)
I20260806 22:47:48.247061 20900 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
Aug 06 22:47:48 dist-test-slave-zh2d krb5kdc[20547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056468, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.66@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
I20260806 22:47:48.255309 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 20900
I20260806 22:47:48.255422 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/minicluster-data/ts-1/wal/instance
I20260806 22:47:48.255671 6305 external_mini_cluster.cc:1428] Setting key f774f64f36d23bf6e6241b542963fbb2
I20260806 22:47:48.257544 20675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:48.247836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.66:59283 (local address 127.6.40.126:40225)
0806 22:47:48.247886 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:48.247888 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:48.248132 (+ 244us) server_negotiation.cc:407] Connection header received
0806 22:47:48.248795 (+ 663us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:48.248796 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:48.248810 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:48.248897 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:48.249914 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:48.250453 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:48.251520 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:48.251745 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:48.251838 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:48.254291 (+ 2453us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:48.254292 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:48.254292 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:48.254300 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:48.255809 (+ 1509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:48.256356 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:48.256357 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:48.256358 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:48.256436 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:48.256721 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:48.256722 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:48.256723 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:48.256921 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:48.256993 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:48.257279 (+ 286us) server_negotiation.cc:299] Negotiation successful
0806 22:47:48.257395 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 22:47:48.257555 21077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:48.247799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:40225 (local address 127.6.40.66:59283)
0806 22:47:48.247999 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:48.248040 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:48.248673 (+ 633us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:48.248981 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:48.248986 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:48.249596 (+ 610us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:48.249803 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:48.249809 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:48.250566 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:48.250567 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:48.251417 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:48.251419 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:48.251853 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:48.251854 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:48.251994 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:48.252604 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:48.252613 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:48.254197 (+ 1584us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:47:48.255927 (+ 1730us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:48.255936 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:48.255938 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:48.256270 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:48.256504 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:48.256505 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:48.256506 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:48.256647 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:48.256998 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:47:48.257004 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:47:48.257145 (+ 141us) client_negotiation.cc:769] Sending connection context
0806 22:47:48.257235 (+ 90us) client_negotiation.cc:240] Negotiation successful
0806 22:47:48.257369 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":100,"threads_started":1}
I20260806 22:47:48.258190 21074 heartbeater.cc:347] Connected to a master server at 127.6.40.126:40225
I20260806 22:47:48.258324 21074 heartbeater.cc:464] Registering TS with master...
I20260806 22:47:48.258556 21074 heartbeater.cc:511] Master 127.6.40.126:40225 requested a full tablet report, sending...
I20260806 22:47:48.259090 20614 ts_manager.cc:194] Registered new tserver with Master: e26d7748ce7747148c2c3ce204b73166 (127.6.40.66:37189)
I20260806 22:47:48.259487 20614 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.40.66@KRBTEST.COM'} at 127.6.40.66:59283
WARNING: no policy specified for kudu/127.6.40.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.40.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.40.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.67.keytab.
Entry for principal kudu/127.6.40.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.67.keytab.
I20260806 22:47:48.283975 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.67:0
--local_ip_for_outbound_sockets=127.6.40.67
--webserver_interface=127.6.40.67
--webserver_port=0
--tserver_master_addrs=127.6.40.126:40225
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.67.keytab
--principal=kudu/127.6.40.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:40519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:47:48.369967 21082 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:47:48.370241 21082 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:47:48.370283 21082 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:47:48.371719 21082 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:47:48.371784 21082 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:47:48.371829 21082 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.67
I20260806 22:47:48.373377 21082 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:40519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.67.keytab
--principal=kudu/127.6.40.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.40.67
--webserver_port=0
--tserver_master_addrs=127.6.40.126:40225
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21082
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.67
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:47:48.373669 21082 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:47:48.373911 21082 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:47:48.379274 21088 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:48.379274 21093 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:48.379503 21082 server_base.cc:1034] running on GCE node
W20260806 22:47:48.379268 21091 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:48.379819 21082 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:47:48.380038 21082 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:47:48.381173 21082 hybrid_clock.cc:648] HybridClock initialized: now 1786056468381164 us; error 31 us; skew 500 ppm
Aug 06 22:47:48 dist-test-slave-zh2d krb5kdc[20547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056468, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:47:48.383739 21082 init.cc:377] Logged in from keytab as kudu/127.6.40.67@KRBTEST.COM (short username kudu)
I20260806 22:47:48.384194 21082 webserver.cc:460] Webserver started at http://127.6.40.67:45323/ using document root <none> and password file <none>
I20260806 22:47:48.384459 21082 fs_manager.cc:359] Metadata directory not provided
I20260806 22:47:48.384526 21082 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:47:48.384684 21082 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:47:48.385820 21082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/minicluster-data/ts-2/data/instance:
uuid: "a00fb7d485f44aaa8e40f08ddab0e7cf"
format_stamp: "Formatted at 2026-08-06 22:47:48 on dist-test-slave-zh2d"
server_key: "dabfb293c0d2c2f3edc1e7a6c315fa93"
server_key_iv: "b1b17c5c66790047cd85bfef2465cf67"
server_key_version: "encryptionkey@0"
I20260806 22:47:48.386199 21082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/minicluster-data/ts-2/wal/instance:
uuid: "a00fb7d485f44aaa8e40f08ddab0e7cf"
format_stamp: "Formatted at 2026-08-06 22:47:48 on dist-test-slave-zh2d"
server_key: "dabfb293c0d2c2f3edc1e7a6c315fa93"
server_key_iv: "b1b17c5c66790047cd85bfef2465cf67"
server_key_version: "encryptionkey@0"
I20260806 22:47:48.387580 21082 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:47:48.388316 21105 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:48.388489 21082 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:47:48.388592 21082 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/minicluster-data/ts-2/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/minicluster-data/ts-2/wal
uuid: "a00fb7d485f44aaa8e40f08ddab0e7cf"
format_stamp: "Formatted at 2026-08-06 22:47:48 on dist-test-slave-zh2d"
server_key: "dabfb293c0d2c2f3edc1e7a6c315fa93"
server_key_iv: "b1b17c5c66790047cd85bfef2465cf67"
server_key_version: "encryptionkey@0"
I20260806 22:47:48.388677 21082 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:47:48.397229 21082 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:47:48.399417 21082 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:47:48.399591 21082 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:47:48.399843 21082 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:47:48.400188 21082 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:47:48.400247 21082 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:48.400290 21082 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:47:48.400318 21082 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:48.406937 21082 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.67:32979
I20260806 22:47:48.406987 21269 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.67:32979 every 8 connection(s)
I20260806 22:47:48.407320 21082 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
I20260806 22:47:48.409188 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 21082
I20260806 22:47:48.409304 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/minicluster-data/ts-2/wal/instance
I20260806 22:47:48.409493 6305 external_mini_cluster.cc:1428] Setting key f09598b9eaf8e8d9c7ebcd8ce93fd0b9
Aug 06 22:47:48 dist-test-slave-zh2d krb5kdc[20547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056468, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.67@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
I20260806 22:47:48.417236 20675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:48.408041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.67:54355 (local address 127.6.40.126:40225)
0806 22:47:48.408097 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:48.408099 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:48.408298 (+ 199us) server_negotiation.cc:407] Connection header received
0806 22:47:48.408934 (+ 636us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:48.408936 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:48.408951 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:48.409015 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:48.410281 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:48.410873 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:48.411954 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:48.412150 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:48.412206 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:48.414492 (+ 2286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:48.414493 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:48.414493 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:48.414501 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:48.415810 (+ 1309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:48.416173 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:48.416174 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:48.416174 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:48.416227 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:48.416515 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:48.416516 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:48.416517 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:48.416646 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:48.416730 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:48.417057 (+ 327us) server_negotiation.cc:299] Negotiation successful
0806 22:47:48.417143 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 22:47:48.417485 21275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:48.407986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:40225 (local address 127.6.40.67:54355)
0806 22:47:48.408152 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:48.408202 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:48.408779 (+ 577us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:48.409088 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:48.409092 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:48.409896 (+ 804us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:48.410188 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:48.410196 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:48.410961 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:48.410962 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:48.411878 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:48.411881 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:48.412226 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:48.412227 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:48.412368 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:48.412959 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:48.412968 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:48.414372 (+ 1404us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:47:48.415883 (+ 1511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:48.415891 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:48.415893 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:48.416107 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:48.416289 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:48.416289 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:48.416290 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:48.416437 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:48.416720 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:47:48.416727 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:47:48.416911 (+ 184us) client_negotiation.cc:769] Sending connection context
0806 22:47:48.417031 (+ 120us) client_negotiation.cc:240] Negotiation successful
0806 22:47:48.417194 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":82,"threads_started":1}
I20260806 22:47:48.417979 21272 heartbeater.cc:347] Connected to a master server at 127.6.40.126:40225
I20260806 22:47:48.418092 21272 heartbeater.cc:464] Registering TS with master...
I20260806 22:47:48.418324 21272 heartbeater.cc:511] Master 127.6.40.126:40225 requested a full tablet report, sending...
I20260806 22:47:48.418694 20614 ts_manager.cc:194] Registered new tserver with Master: a00fb7d485f44aaa8e40f08ddab0e7cf (127.6.40.67:32979)
I20260806 22:47:48.419103 20614 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.40.67@KRBTEST.COM'} at 127.6.40.67:54355
I20260806 22:47:48.421265 6305 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:47:48.429265 20675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:48.422845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49166 (local address 127.6.40.126:40225)
0806 22:47:48.422924 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:48.422927 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:48.423055 (+ 128us) server_negotiation.cc:407] Connection header received
0806 22:47:48.423146 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:48.423147 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:48.423158 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:48.423237 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:48.423871 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:48.424401 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:48.425307 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:48.425529 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:48.425636 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:48.426865 (+ 1229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:48.426866 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:48.426867 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:48.426874 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:48.428033 (+ 1159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:48.428385 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:48.428386 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:48.428387 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:48.428447 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:48.428675 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:48.428676 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:48.428676 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:48.428824 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:48.428963 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:48.429084 (+ 121us) server_negotiation.cc:299] Negotiation successful
0806 22:47:48.429166 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 22:47:48.430722 20614 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49166:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:47:48.432363 20614 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:47:48.439213 21288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:48.434812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.66:37189 (local address 127.0.0.1:40734)
0806 22:47:48.435265 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:48.435271 (+ 6us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:48.435295 (+ 24us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:48.435605 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:48.435605 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:48.435607 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:47:48.435649 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:48.435650 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:48.437067 (+ 1417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:48.437068 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:48.438183 (+ 1115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:48.438186 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:48.438953 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:48.438953 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:48.439000 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:48.439005 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:47:48.439052 (+ 47us) client_negotiation.cc:240] Negotiation successful
0806 22:47:48.439110 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":444,"mutex_wait_us":42,"thread_start_us":98,"threads_started":1}
I20260806 22:47:48.439213 21287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:48.434799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.67:32979 (local address 127.0.0.1:36270)
0806 22:47:48.434982 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:48.435020 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:48.435091 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:48.435515 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:48.435520 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:48.435523 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:47:48.435606 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:48.435608 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:48.436936 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:48.436937 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:48.438058 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:48.438061 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:48.438903 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:48.438905 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:48.438988 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:48.439004 (+ 16us) client_negotiation.cc:769] Sending connection context
0806 22:47:48.439087 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 22:47:48.439117 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"mutex_wait_us":84,"thread_start_us":47,"threads_started":1}
I20260806 22:47:48.439254 21290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:48.434878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36270 (local address 127.6.40.67:32979)
0806 22:47:48.435318 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:48.435323 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:48.435334 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:47:48.435376 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:48.435378 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:48.435395 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:48.435495 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:47:48.435694 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:48.436824 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:48.438148 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:48.438805 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:48.438892 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:48.438949 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:48.439119 (+ 170us) server_negotiation.cc:299] Negotiation successful
0806 22:47:48.439167 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":404,"thread_start_us":83,"threads_started":1}
I20260806 22:47:48.439617 21289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:48.434804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.65:46423 (local address 127.0.0.1:58828)
0806 22:47:48.435185 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:48.435192 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:48.435219 (+ 27us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:48.435587 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:48.435588 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:48.435591 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:47:48.435675 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:48.435677 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:48.436860 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:48.436862 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:48.438330 (+ 1468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:48.438332 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:48.439301 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:48.439302 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:48.439434 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:48.439437 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:47:48.439511 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 22:47:48.439552 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":364,"mutex_wait_us":67,"thread_start_us":107,"threads_started":1}
I20260806 22:47:48.439662 21292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:48.435096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58828 (local address 127.6.40.65:46423)
0806 22:47:48.435318 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:48.435323 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:48.435334 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:47:48.435378 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:48.435379 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:48.435395 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:48.435769 (+ 374us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:47:48.435864 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:48.436733 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:48.438426 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:48.439142 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:48.439209 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:48.439262 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:48.439533 (+ 271us) server_negotiation.cc:299] Negotiation successful
0806 22:47:48.439578 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":96,"threads_started":1}
I20260806 22:47:48.439698 21291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:48.435087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40734 (local address 127.6.40.66:37189)
0806 22:47:48.435407 (+ 320us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:48.435412 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:48.435420 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:47:48.435478 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:48.435480 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:48.435498 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:48.435729 (+ 231us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:47:48.435818 (+ 89us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:48.436992 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:48.438278 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:48.438831 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:48.439437 (+ 606us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:48.439523 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:48.439566 (+ 43us) server_negotiation.cc:299] Negotiation successful
0806 22:47:48.439613 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":97,"threads_started":1}
I20260806 22:47:48.440447 20981 tablet_service.cc:1467] Processing CreateTablet for tablet f1693992ce3a48cabf20e047c2496f40 (DEFAULT_TABLE table=test-table [id=aec6d212e48a4f36838ddfbc5b7e0e27]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:47:48.440443 21174 tablet_service.cc:1467] Processing CreateTablet for tablet f1693992ce3a48cabf20e047c2496f40 (DEFAULT_TABLE table=test-table [id=aec6d212e48a4f36838ddfbc5b7e0e27]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:47:48.440817 21174 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f1693992ce3a48cabf20e047c2496f40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:47:48.440819 20981 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f1693992ce3a48cabf20e047c2496f40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:47:48.441257 20797 tablet_service.cc:1467] Processing CreateTablet for tablet f1693992ce3a48cabf20e047c2496f40 (DEFAULT_TABLE table=test-table [id=aec6d212e48a4f36838ddfbc5b7e0e27]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:47:48.441583 20797 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f1693992ce3a48cabf20e047c2496f40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:47:48.444164 21295 tablet_bootstrap.cc:492] T f1693992ce3a48cabf20e047c2496f40 P d1083c944f814031a6268a9fadbc7d78: Bootstrap starting.
I20260806 22:47:48.445020 21296 tablet_bootstrap.cc:492] T f1693992ce3a48cabf20e047c2496f40 P e26d7748ce7747148c2c3ce204b73166: Bootstrap starting.
I20260806 22:47:48.445060 21295 tablet_bootstrap.cc:654] T f1693992ce3a48cabf20e047c2496f40 P d1083c944f814031a6268a9fadbc7d78: Neither blocks nor log segments found. Creating new log.
I20260806 22:47:48.445557 21295 log.cc:824] T f1693992ce3a48cabf20e047c2496f40 P d1083c944f814031a6268a9fadbc7d78: Log is configured to *not* fsync() on all Append() calls
I20260806 22:47:48.445598 21296 tablet_bootstrap.cc:654] T f1693992ce3a48cabf20e047c2496f40 P e26d7748ce7747148c2c3ce204b73166: Neither blocks nor log segments found. Creating new log.
I20260806 22:47:48.446069 21296 log.cc:824] T f1693992ce3a48cabf20e047c2496f40 P e26d7748ce7747148c2c3ce204b73166: Log is configured to *not* fsync() on all Append() calls
I20260806 22:47:48.446363 21295 tablet_bootstrap.cc:492] T f1693992ce3a48cabf20e047c2496f40 P d1083c944f814031a6268a9fadbc7d78: No bootstrap required, opened a new log
I20260806 22:47:48.446388 21298 tablet_bootstrap.cc:492] T f1693992ce3a48cabf20e047c2496f40 P a00fb7d485f44aaa8e40f08ddab0e7cf: Bootstrap starting.
I20260806 22:47:48.446446 21295 ts_tablet_manager.cc:1397] T f1693992ce3a48cabf20e047c2496f40 P d1083c944f814031a6268a9fadbc7d78: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:47:48.446918 21296 tablet_bootstrap.cc:492] T f1693992ce3a48cabf20e047c2496f40 P e26d7748ce7747148c2c3ce204b73166: No bootstrap required, opened a new log
I20260806 22:47:48.446990 21296 ts_tablet_manager.cc:1397] T f1693992ce3a48cabf20e047c2496f40 P e26d7748ce7747148c2c3ce204b73166: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:47:48.447126 21298 tablet_bootstrap.cc:654] T f1693992ce3a48cabf20e047c2496f40 P a00fb7d485f44aaa8e40f08ddab0e7cf: Neither blocks nor log segments found. Creating new log.
I20260806 22:47:48.447693 21298 log.cc:824] T f1693992ce3a48cabf20e047c2496f40 P a00fb7d485f44aaa8e40f08ddab0e7cf: Log is configured to *not* fsync() on all Append() calls
I20260806 22:47:48.448215 21295 raft_consensus.cc:348] T f1693992ce3a48cabf20e047c2496f40 P d1083c944f814031a6268a9fadbc7d78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a00fb7d485f44aaa8e40f08ddab0e7cf" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 32979 } } peers { permanent_uuid: "d1083c944f814031a6268a9fadbc7d78" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 46423 } } peers { permanent_uuid: "e26d7748ce7747148c2c3ce204b73166" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 37189 } }
I20260806 22:47:48.448377 21295 raft_consensus.cc:374] T f1693992ce3a48cabf20e047c2496f40 P d1083c944f814031a6268a9fadbc7d78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:47:48.448421 21295 raft_consensus.cc:729] T f1693992ce3a48cabf20e047c2496f40 P d1083c944f814031a6268a9fadbc7d78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1083c944f814031a6268a9fadbc7d78, State: Initialized, Role: FOLLOWER
I20260806 22:47:48.448503 21298 tablet_bootstrap.cc:492] T f1693992ce3a48cabf20e047c2496f40 P a00fb7d485f44aaa8e40f08ddab0e7cf: No bootstrap required, opened a new log
I20260806 22:47:48.448575 21298 ts_tablet_manager.cc:1397] T f1693992ce3a48cabf20e047c2496f40 P a00fb7d485f44aaa8e40f08ddab0e7cf: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:47:48.448570 21295 consensus_queue.cc:260] T f1693992ce3a48cabf20e047c2496f40 P d1083c944f814031a6268a9fadbc7d78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a00fb7d485f44aaa8e40f08ddab0e7cf" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 32979 } } peers { permanent_uuid: "d1083c944f814031a6268a9fadbc7d78" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 46423 } } peers { permanent_uuid: "e26d7748ce7747148c2c3ce204b73166" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 37189 } }
I20260806 22:47:48.448899 20894 heartbeater.cc:503] Master 127.6.40.126:40225 was elected leader, sending a full tablet report...
I20260806 22:47:48.449187 21295 ts_tablet_manager.cc:1428] T f1693992ce3a48cabf20e047c2496f40 P d1083c944f814031a6268a9fadbc7d78: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:47:48.449147 21296 raft_consensus.cc:348] T f1693992ce3a48cabf20e047c2496f40 P e26d7748ce7747148c2c3ce204b73166 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a00fb7d485f44aaa8e40f08ddab0e7cf" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 32979 } } peers { permanent_uuid: "d1083c944f814031a6268a9fadbc7d78" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 46423 } } peers { permanent_uuid: "e26d7748ce7747148c2c3ce204b73166" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 37189 } }
I20260806 22:47:48.449306 21296 raft_consensus.cc:374] T f1693992ce3a48cabf20e047c2496f40 P e26d7748ce7747148c2c3ce204b73166 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:47:48.449349 21296 raft_consensus.cc:729] T f1693992ce3a48cabf20e047c2496f40 P e26d7748ce7747148c2c3ce204b73166 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e26d7748ce7747148c2c3ce204b73166, State: Initialized, Role: FOLLOWER
I20260806 22:47:48.449476 21296 consensus_queue.cc:260] T f1693992ce3a48cabf20e047c2496f40 P e26d7748ce7747148c2c3ce204b73166 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a00fb7d485f44aaa8e40f08ddab0e7cf" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 32979 } } peers { permanent_uuid: "d1083c944f814031a6268a9fadbc7d78" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 46423 } } peers { permanent_uuid: "e26d7748ce7747148c2c3ce204b73166" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 37189 } }
I20260806 22:47:48.449740 21296 ts_tablet_manager.cc:1428] T f1693992ce3a48cabf20e047c2496f40 P e26d7748ce7747148c2c3ce204b73166: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:47:48.449863 21074 heartbeater.cc:503] Master 127.6.40.126:40225 was elected leader, sending a full tablet report...
I20260806 22:47:48.450722 21298 raft_consensus.cc:348] T f1693992ce3a48cabf20e047c2496f40 P a00fb7d485f44aaa8e40f08ddab0e7cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a00fb7d485f44aaa8e40f08ddab0e7cf" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 32979 } } peers { permanent_uuid: "d1083c944f814031a6268a9fadbc7d78" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 46423 } } peers { permanent_uuid: "e26d7748ce7747148c2c3ce204b73166" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 37189 } }
I20260806 22:47:48.450896 21298 raft_consensus.cc:374] T f1693992ce3a48cabf20e047c2496f40 P a00fb7d485f44aaa8e40f08ddab0e7cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:47:48.450924 21298 raft_consensus.cc:729] T f1693992ce3a48cabf20e047c2496f40 P a00fb7d485f44aaa8e40f08ddab0e7cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a00fb7d485f44aaa8e40f08ddab0e7cf, State: Initialized, Role: FOLLOWER
I20260806 22:47:48.451025 21298 consensus_queue.cc:260] T f1693992ce3a48cabf20e047c2496f40 P a00fb7d485f44aaa8e40f08ddab0e7cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a00fb7d485f44aaa8e40f08ddab0e7cf" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 32979 } } peers { permanent_uuid: "d1083c944f814031a6268a9fadbc7d78" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 46423 } } peers { permanent_uuid: "e26d7748ce7747148c2c3ce204b73166" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 37189 } }
I20260806 22:47:48.451256 21298 ts_tablet_manager.cc:1428] T f1693992ce3a48cabf20e047c2496f40 P a00fb7d485f44aaa8e40f08ddab0e7cf: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:47:48.451323 21272 heartbeater.cc:503] Master 127.6.40.126:40225 was elected leader, sending a full tablet report...
I20260806 22:47:48.478389 21304 raft_consensus.cc:482] T f1693992ce3a48cabf20e047c2496f40 P a00fb7d485f44aaa8e40f08ddab0e7cf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:47:48.478551 21304 raft_consensus.cc:504] T f1693992ce3a48cabf20e047c2496f40 P a00fb7d485f44aaa8e40f08ddab0e7cf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a00fb7d485f44aaa8e40f08ddab0e7cf" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 32979 } } peers { permanent_uuid: "d1083c944f814031a6268a9fadbc7d78" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 46423 } } peers { permanent_uuid: "e26d7748ce7747148c2c3ce204b73166" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 37189 } }
I20260806 22:47:48.479002 21304 leader_election.cc:288] T f1693992ce3a48cabf20e047c2496f40 P a00fb7d485f44aaa8e40f08ddab0e7cf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d1083c944f814031a6268a9fadbc7d78 (127.6.40.65:46423), e26d7748ce7747148c2c3ce204b73166 (127.6.40.66:37189)
I20260806 22:47:48.482110 21292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:48.479261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.67:39121 (local address 127.6.40.65:46423)
0806 22:47:48.479306 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:48.479308 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:48.479316 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:47:48.479344 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:48.479345 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:48.479358 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:48.479421 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:47:48.479667 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:48.480210 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:48.481251 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:48.481697 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:48.481754 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:48.481765 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:48.481981 (+ 216us) server_negotiation.cc:299] Negotiation successful
0806 22:47:48.482010 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 22:47:48.482110 21275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:48.479152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.65:46423 (local address 127.6.40.67:39121)
0806 22:47:48.479211 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:48.479223 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:48.479266 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:48.479453 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:48.479455 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:48.479489 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:47:48.479577 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:48.479579 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:48.480297 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:48.480298 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:48.481163 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:48.481166 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:48.481791 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:48.481792 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:48.481892 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:48.481897 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:47:48.481969 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 22:47:48.482018 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":25}
I20260806 22:47:48.482398 21291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:48.479306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.67:48223 (local address 127.6.40.66:37189)
0806 22:47:48.479358 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:48.479361 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:48.479504 (+ 143us) server_negotiation.cc:407] Connection header received
0806 22:47:48.479539 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:48.479541 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:48.479558 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:48.479628 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:47:48.479818 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:48.480329 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:48.481456 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:48.481999 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:48.482067 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:48.482078 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:48.482279 (+ 201us) server_negotiation.cc:299] Negotiation successful
0806 22:47:48.482330 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 22:47:48.482623 21306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:48.479219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.66:37189 (local address 127.6.40.67:48223)
0806 22:47:48.479411 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:48.479421 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:48.479473 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:48.479663 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:48.479664 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:48.479665 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:47:48.479742 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:48.479743 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:48.480460 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:48.480461 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:48.481346 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:48.481348 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:48.482085 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:48.482086 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:48.482166 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:48.482170 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:47:48.482484 (+ 314us) client_negotiation.cc:240] Negotiation successful
0806 22:47:48.482544 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":119,"threads_started":1}
I20260806 22:47:48.482577 20828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1693992ce3a48cabf20e047c2496f40" candidate_uuid: "a00fb7d485f44aaa8e40f08ddab0e7cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1083c944f814031a6268a9fadbc7d78" is_pre_election: true
I20260806 22:47:48.482789 20828 raft_consensus.cc:2454] T f1693992ce3a48cabf20e047c2496f40 P d1083c944f814031a6268a9fadbc7d78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a00fb7d485f44aaa8e40f08ddab0e7cf in term 0.
I20260806 22:47:48.483105 21114 leader_election.cc:302] T f1693992ce3a48cabf20e047c2496f40 P a00fb7d485f44aaa8e40f08ddab0e7cf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a00fb7d485f44aaa8e40f08ddab0e7cf, d1083c944f814031a6268a9fadbc7d78; no voters:
I20260806 22:47:48.483120 21009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1693992ce3a48cabf20e047c2496f40" candidate_uuid: "a00fb7d485f44aaa8e40f08ddab0e7cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e26d7748ce7747148c2c3ce204b73166" is_pre_election: true
I20260806 22:47:48.483239 21304 raft_consensus.cc:2781] T f1693992ce3a48cabf20e047c2496f40 P a00fb7d485f44aaa8e40f08ddab0e7cf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:47:48.483311 21009 raft_consensus.cc:2454] T f1693992ce3a48cabf20e047c2496f40 P e26d7748ce7747148c2c3ce204b73166 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a00fb7d485f44aaa8e40f08ddab0e7cf in term 0.
I20260806 22:47:48.483330 21304 raft_consensus.cc:482] T f1693992ce3a48cabf20e047c2496f40 P a00fb7d485f44aaa8e40f08ddab0e7cf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:47:48.483356 21304 raft_consensus.cc:3037] T f1693992ce3a48cabf20e047c2496f40 P a00fb7d485f44aaa8e40f08ddab0e7cf [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:47:48.484182 21304 raft_consensus.cc:504] T f1693992ce3a48cabf20e047c2496f40 P a00fb7d485f44aaa8e40f08ddab0e7cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a00fb7d485f44aaa8e40f08ddab0e7cf" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 32979 } } peers { permanent_uuid: "d1083c944f814031a6268a9fadbc7d78" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 46423 } } peers { permanent_uuid: "e26d7748ce7747148c2c3ce204b73166" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 37189 } }
I20260806 22:47:48.484398 21304 leader_election.cc:288] T f1693992ce3a48cabf20e047c2496f40 P a00fb7d485f44aaa8e40f08ddab0e7cf [CANDIDATE]: Term 1 election: Requested vote from peers d1083c944f814031a6268a9fadbc7d78 (127.6.40.65:46423), e26d7748ce7747148c2c3ce204b73166 (127.6.40.66:37189)
I20260806 22:47:48.484486 21009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1693992ce3a48cabf20e047c2496f40" candidate_uuid: "a00fb7d485f44aaa8e40f08ddab0e7cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e26d7748ce7747148c2c3ce204b73166"
I20260806 22:47:48.484511 20828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1693992ce3a48cabf20e047c2496f40" candidate_uuid: "a00fb7d485f44aaa8e40f08ddab0e7cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1083c944f814031a6268a9fadbc7d78"
I20260806 22:47:48.484586 21009 raft_consensus.cc:3037] T f1693992ce3a48cabf20e047c2496f40 P e26d7748ce7747148c2c3ce204b73166 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:47:48.484591 20828 raft_consensus.cc:3037] T f1693992ce3a48cabf20e047c2496f40 P d1083c944f814031a6268a9fadbc7d78 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:47:48.485420 20828 raft_consensus.cc:2454] T f1693992ce3a48cabf20e047c2496f40 P d1083c944f814031a6268a9fadbc7d78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a00fb7d485f44aaa8e40f08ddab0e7cf in term 1.
I20260806 22:47:48.485430 21009 raft_consensus.cc:2454] T f1693992ce3a48cabf20e047c2496f40 P e26d7748ce7747148c2c3ce204b73166 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a00fb7d485f44aaa8e40f08ddab0e7cf in term 1.
I20260806 22:47:48.485631 21114 leader_election.cc:302] T f1693992ce3a48cabf20e047c2496f40 P a00fb7d485f44aaa8e40f08ddab0e7cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a00fb7d485f44aaa8e40f08ddab0e7cf, d1083c944f814031a6268a9fadbc7d78; no voters:
I20260806 22:47:48.485744 21304 raft_consensus.cc:2781] T f1693992ce3a48cabf20e047c2496f40 P a00fb7d485f44aaa8e40f08ddab0e7cf [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:47:48.485929 21304 raft_consensus.cc:686] T f1693992ce3a48cabf20e047c2496f40 P a00fb7d485f44aaa8e40f08ddab0e7cf [term 1 LEADER]: Becoming Leader. State: Replica: a00fb7d485f44aaa8e40f08ddab0e7cf, State: Running, Role: LEADER
I20260806 22:47:48.486053 21304 consensus_queue.cc:237] T f1693992ce3a48cabf20e047c2496f40 P a00fb7d485f44aaa8e40f08ddab0e7cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a00fb7d485f44aaa8e40f08ddab0e7cf" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 32979 } } peers { permanent_uuid: "d1083c944f814031a6268a9fadbc7d78" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 46423 } } peers { permanent_uuid: "e26d7748ce7747148c2c3ce204b73166" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 37189 } }
I20260806 22:47:48.486924 20612 catalog_manager.cc:5495] T f1693992ce3a48cabf20e047c2496f40 P a00fb7d485f44aaa8e40f08ddab0e7cf reported cstate change: term changed from 0 to 1, leader changed from <none> to a00fb7d485f44aaa8e40f08ddab0e7cf (127.6.40.67). New cstate: current_term: 1 leader_uuid: "a00fb7d485f44aaa8e40f08ddab0e7cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a00fb7d485f44aaa8e40f08ddab0e7cf" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 32979 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1083c944f814031a6268a9fadbc7d78" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 46423 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e26d7748ce7747148c2c3ce204b73166" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 37189 } health_report { overall_health: UNKNOWN } } }
W20260806 22:47:48.497681 21075 tablet.cc:2369] T f1693992ce3a48cabf20e047c2496f40 P e26d7748ce7747148c2c3ce204b73166: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:47:48.504485 21290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:48.502175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36272 (local address 127.6.40.67:32979)
0806 22:47:48.502241 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:48.502243 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:48.502259 (+ 16us) server_negotiation.cc:407] Connection header received
0806 22:47:48.502296 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:48.502297 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:48.502317 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:48.502381 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:47:48.502516 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:48.502996 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:48.503740 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:48.503948 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:48.504011 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:48.504150 (+ 139us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:47:48.504256 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:47:48.504320 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:48.504353 (+ 33us) server_negotiation.cc:299] Negotiation successful
0806 22:47:48.504373 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 22:47:48.506551 20828 raft_consensus.cc:1261] T f1693992ce3a48cabf20e047c2496f40 P d1083c944f814031a6268a9fadbc7d78 [term 1 FOLLOWER]: Refusing update from remote peer a00fb7d485f44aaa8e40f08ddab0e7cf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:47:48.506551 21009 raft_consensus.cc:1261] T f1693992ce3a48cabf20e047c2496f40 P e26d7748ce7747148c2c3ce204b73166 [term 1 FOLLOWER]: Refusing update from remote peer a00fb7d485f44aaa8e40f08ddab0e7cf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:47:48.506982 21304 consensus_queue.cc:1035] T f1693992ce3a48cabf20e047c2496f40 P a00fb7d485f44aaa8e40f08ddab0e7cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "e26d7748ce7747148c2c3ce204b73166" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 37189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:47:48.507149 21307 consensus_queue.cc:1035] T f1693992ce3a48cabf20e047c2496f40 P a00fb7d485f44aaa8e40f08ddab0e7cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1083c944f814031a6268a9fadbc7d78" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 46423 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:47:48.509509 21312 mvcc.cc:205] Tried to move back new op lower bound from 7315687294999744512 to 7315687294919606272. Current Snapshot: MvccSnapshot[applied={T|T < 7315687294999744512}]
I20260806 22:47:48.509752 21313 mvcc.cc:205] Tried to move back new op lower bound from 7315687294999744512 to 7315687294919606272. Current Snapshot: MvccSnapshot[applied={T|T < 7315687294999744512}]
I20260806 22:47:48.509863 21314 mvcc.cc:205] Tried to move back new op lower bound from 7315687294999744512 to 7315687294919606272. Current Snapshot: MvccSnapshot[applied={T|T < 7315687294999744512}]
I20260806 22:47:48.512259 20614 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49166:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:47:48.512357 20614 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49166:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:47:48.513227 20614 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 036aefa83777434fb51d359c1e235b0b: Sending DeleteTablet for 3 replicas of tablet f1693992ce3a48cabf20e047c2496f40
I20260806 22:47:48.513597 21174 tablet_service.cc:1514] Processing DeleteTablet for tablet f1693992ce3a48cabf20e047c2496f40 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:47:48 UTC) from {username='kudu', principal='kudu/127.6.40.126@KRBTEST.COM'} at 127.0.0.1:36270
I20260806 22:47:48.513621 20797 tablet_service.cc:1514] Processing DeleteTablet for tablet f1693992ce3a48cabf20e047c2496f40 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:47:48 UTC) from {username='kudu', principal='kudu/127.6.40.126@KRBTEST.COM'} at 127.0.0.1:58828
I20260806 22:47:48.513695 20981 tablet_service.cc:1514] Processing DeleteTablet for tablet f1693992ce3a48cabf20e047c2496f40 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:47:48 UTC) from {username='kudu', principal='kudu/127.6.40.126@KRBTEST.COM'} at 127.0.0.1:40734
I20260806 22:47:48.513914 21327 tablet_replica.cc:330] T f1693992ce3a48cabf20e047c2496f40 P a00fb7d485f44aaa8e40f08ddab0e7cf: stopping tablet replica
I20260806 22:47:48.514062 21327 raft_consensus.cc:2229] T f1693992ce3a48cabf20e047c2496f40 P a00fb7d485f44aaa8e40f08ddab0e7cf [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:47:48.514175 21327 raft_consensus.cc:2258] T f1693992ce3a48cabf20e047c2496f40 P a00fb7d485f44aaa8e40f08ddab0e7cf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:47:48.514310 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 20710
I20260806 22:47:48.514340 21332 tablet_replica.cc:330] T f1693992ce3a48cabf20e047c2496f40 P d1083c944f814031a6268a9fadbc7d78: stopping tablet replica
I20260806 22:47:48.521693 21327 ts_tablet_manager.cc:1905] T f1693992ce3a48cabf20e047c2496f40 P a00fb7d485f44aaa8e40f08ddab0e7cf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:47:48.523730 21327 ts_tablet_manager.cc:1918] T f1693992ce3a48cabf20e047c2496f40 P a00fb7d485f44aaa8e40f08ddab0e7cf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:47:48.523820 21327 log.cc:1196] T f1693992ce3a48cabf20e047c2496f40 P a00fb7d485f44aaa8e40f08ddab0e7cf: Deleting WAL directory at /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056405724609-6305-0/minicluster-data/ts-2/wal/wals/f1693992ce3a48cabf20e047c2496f40
I20260806 22:47:48.524132 21327 ts_tablet_manager.cc:1939] T f1693992ce3a48cabf20e047c2496f40 P a00fb7d485f44aaa8e40f08ddab0e7cf: Deleting consensus metadata
I20260806 22:47:48.524603 20588 catalog_manager.cc:4841] TS a00fb7d485f44aaa8e40f08ddab0e7cf (127.6.40.67:32979): tablet f1693992ce3a48cabf20e047c2496f40 (table test-table [id=aec6d212e48a4f36838ddfbc5b7e0e27]) successfully deleted
W20260806 22:47:48.526117 20592 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.40.65:46423 (error 108)
I20260806 22:47:48.526718 21289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:48.526568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.65:46423 (local address 127.0.0.1:58838)
0806 22:47:48.526633 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:48.526677 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.65:46423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260806 22:47:48.526798 21333 tablet_replica.cc:330] T f1693992ce3a48cabf20e047c2496f40 P e26d7748ce7747148c2c3ce204b73166: stopping tablet replica
I20260806 22:47:48.526919 21333 raft_consensus.cc:2229] T f1693992ce3a48cabf20e047c2496f40 P e26d7748ce7747148c2c3ce204b73166 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260806 22:47:48.526895 20592 catalog_manager.cc:4584] TS d1083c944f814031a6268a9fadbc7d78 (127.6.40.65:46423): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f1693992ce3a48cabf20e047c2496f40: Network error: Client connection negotiation failed: client connection to 127.6.40.65:46423: connect: Connection refused (error 111)
I20260806 22:47:48.527015 21333 raft_consensus.cc:2258] T f1693992ce3a48cabf20e047c2496f40 P e26d7748ce7747148c2c3ce204b73166 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:47:48.527212 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 20900
I20260806 22:47:48.542402 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 21082
I20260806 22:47:48.542583 21289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:48.542386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.66:37189 (local address 127.0.0.1:40750)
0806 22:47:48.542486 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:48.542534 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.66:37189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260806 22:47:48.551690 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 20566
2026-08-06T22:47:48Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2231 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:47:48 dist-test-slave-zh2d krb5kdc[21338](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:47:48 dist-test-slave-zh2d krb5kdc[21338](info): set up 2 sockets
Aug 06 22:47:48 dist-test-slave-zh2d krb5kdc[21338](info): commencing operation
krb5kdc: starting...
W20260806 22:47:49.840818 6305 mini_kdc.cc:121] Time spent starting KDC: real 1.265s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/krb5kdc/test-user.keytab.
Aug 06 22:47:49 dist-test-slave-zh2d krb5kdc[21338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056469, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:47:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:47:49Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.6.40.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.6.40.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.6.40.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/krb5kdc/rangeradmin_127.6.40.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.6.40.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/krb5kdc/rangeradmin_127.6.40.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.6.40.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.6.40.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.6.40.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/krb5kdc/rangerlookup_127.6.40.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.6.40.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/krb5kdc/rangerlookup_127.6.40.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.6.40.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.6.40.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.6.40.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/krb5kdc/HTTP_127.6.40.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.6.40.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/krb5kdc/HTTP_127.6.40.84@KRBTEST.COM.keytab.
I20260806 22:47:50.016477 6305 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task_hhfSb/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/minicluster-data/postgres -l logfile start
2026-08-06 22:47:52.687 UTC [21382] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-06 22:47:52.687 UTC [21382] LOG: listening on IPv4 address "127.6.40.84", port 42801
2026-08-06 22:47:52.691 UTC [21382] LOG: listening on Unix socket "/tmp/.s.PGSQL.42801"
2026-08-06 22:47:52.714 UTC [21386] LOG: database system was shut down at 2026-08-06 22:47:51 UTC
2026-08-06 22:47:52.718 UTC [21382] LOG: database system is ready to accept connections
I20260806 22:47:53.853359 6305 mini_postgres.cc:96] Postgres bound to 42801
2026-08-06 22:47:53.861 UTC [21397] FATAL: database "slave" does not exist
127.6.40.84:42801 - accepting connections
I20260806 22:47:53.862262 6305 mini_ranger.cc:161] Starting Ranger...
I20260806 22:47:53.878096 6305 mini_ranger.cc:84] Created miniranger Postgres user
I20260806 22:47:54.435060 6305 mini_ranger.cc:87] Created ranger Postgres database
I20260806 22:47:54.435149 6305 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/minicluster-data/ranger-admin
2026-08-06 22:47:54,776 [I] DB FLAVOR :POSTGRES
2026-08-06 22:47:54,777 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 22:47:54,777 [I] Checking connection..
2026-08-06 22:47:54,777 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_hhfSb/build/release/bin/postgresql.jar:/tmp/dist-test-task_hhfSb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.40.84:42801/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-06 22:47:55,053 [I] Checking connection passed.
2026-08-06 22:47:55,053 [I] --------- Verifying version history table ---------
2026-08-06 22:47:55,053 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_hhfSb/build/release/bin/postgresql.jar:/tmp/dist-test-task_hhfSb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.40.84:42801/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 22:47:55,332 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 22:47:55,333 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_hhfSb/build/release/bin/postgresql.jar:/tmp/dist-test-task_hhfSb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.40.84:42801/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 22:47:55,592 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 22:47:55,592 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-06 22:47:55,592 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_hhfSb/build/release/bin/postgresql.jar:/tmp/dist-test-task_hhfSb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.40.84:42801/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task_hhfSb/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-06 22:47:55.859 UTC [21491] WARNING: there is no transaction in progress
2026-08-06 22:47:55,869 [I] create_dbversion_catalog.sql file imported successfully
2026-08-06 22:47:55,869 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_hhfSb/build/release/bin/postgresql.jar:/tmp/dist-test-task_hhfSb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.40.84:42801/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 22:47:56,142 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-06 22:47:56,142 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 22:47:56,143 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_hhfSb/build/release/bin/postgresql.jar:/tmp/dist-test-task_hhfSb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.40.84:42801/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-06 22:47:56,402 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_hhfSb/build/release/bin/postgresql.jar:/tmp/dist-test-task_hhfSb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.40.84:42801/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-06 22:47:56,677 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_hhfSb/build/release/bin/postgresql.jar:/tmp/dist-test-task_hhfSb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.40.84:42801/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-zh2d.c.gcp-upstream.internal','N') ;"
2026-08-06 22:47:56,952 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_hhfSb/build/release/bin/postgresql.jar:/tmp/dist-test-task_hhfSb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.40.84:42801/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 22:47:57,243 [I] Table x_portal_user does not exist in database ranger
2026-08-06 22:47:57,243 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_hhfSb/build/release/bin/postgresql.jar:/tmp/dist-test-task_hhfSb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.40.84:42801/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 22:47:57,518 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-06 22:47:57,519 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_hhfSb/build/release/bin/postgresql.jar:/tmp/dist-test-task_hhfSb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.40.84:42801/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 22:47:57,799 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_hhfSb/build/release/bin/postgresql.jar:/tmp/dist-test-task_hhfSb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.40.84:42801/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 22:47:58,044 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-06 22:47:58,045 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_hhfSb/build/release/bin/postgresql.jar:/tmp/dist-test-task_hhfSb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.40.84:42801/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task_hhfSb/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-06 22:47:59.103 UTC [21638] WARNING: there is no transaction in progress
2026-08-06 22:47:59.122 UTC [21638] WARNING: there is no transaction in progress
2026-08-06 22:47:59.141 UTC [21638] WARNING: there is no transaction in progress
2026-08-06 22:47:59.157 UTC [21638] WARNING: there is no transaction in progress
2026-08-06 22:47:59.174 UTC [21638] WARNING: there is no transaction in progress
2026-08-06 22:47:59.196 UTC [21638] WARNING: there is no transaction in progress
2026-08-06 22:47:59.256 UTC [21638] WARNING: there is no transaction in progress
2026-08-06 22:47:59.264 UTC [21638] WARNING: there is no transaction in progress
2026-08-06 22:47:59.280 UTC [21638] WARNING: there is no transaction in progress
2026-08-06 22:47:59.290 UTC [21638] WARNING: there is no transaction in progress
2026-08-06 22:47:59.301 UTC [21638] WARNING: there is no transaction in progress
2026-08-06 22:47:59.317 UTC [21638] WARNING: there is no transaction in progress
2026-08-06 22:47:59.328 UTC [21638] WARNING: there is no transaction in progress
2026-08-06 22:47:59.335 UTC [21638] WARNING: there is no transaction in progress
2026-08-06 22:47:59.343 UTC [21638] WARNING: there is no transaction in progress
2026-08-06 22:47:59.351 UTC [21638] WARNING: there is no transaction in progress
2026-08-06 22:48:00,098 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-06 22:48:00,098 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_hhfSb/build/release/bin/postgresql.jar:/tmp/dist-test-task_hhfSb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.40.84:42801/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-06 22:48:00,375 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-06 22:48:00,376 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_hhfSb/build/release/bin/postgresql.jar:/tmp/dist-test-task_hhfSb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.40.84:42801/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 22:48:00,672 [I] Table x_portal_user already exists in database 'ranger'
2026-08-06 22:48:00,672 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_hhfSb/build/release/bin/postgresql.jar:/tmp/dist-test-task_hhfSb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.40.84:42801/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 22:48:00,964 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-06 22:48:00,965 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_hhfSb/build/release/bin/postgresql.jar:/tmp/dist-test-task_hhfSb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.40.84:42801/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 22:48:01,237 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_hhfSb/build/release/bin/postgresql.jar:/tmp/dist-test-task_hhfSb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.40.84:42801/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 22:48:01,504 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_hhfSb/build/release/bin/postgresql.jar:/tmp/dist-test-task_hhfSb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.40.84:42801/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-zh2d.c.gcp-upstream.internal';"
2026-08-06 22:48:01,775 [I] CORE_DB_SCHEMA import status has been updated
2026-08-06 22:48:01,775 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_hhfSb/build/release/bin/postgresql.jar:/tmp/dist-test-task_hhfSb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.40.84:42801/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-06 22:48:02,033 [I] DB_PATCHES have already been applied
I20260806 22:48:02.039256 6305 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/minicluster-data/ranger-admin:/tmp/dist-test-task_hhfSb/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task_hhfSb/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task_hhfSb/build/release/bin/postgresql.jar:/tmp/dist-test-task_hhfSb/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260806 22:48:02.039346 6305 mini_ranger.cc:193] Using host: 127.6.40.84
I20260806 22:48:02.043329 6305 mini_ranger.cc:227] Ranger admin URL: http://127.6.40.84:33067
Aug 06, 2026 10:48:02 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 10:48:02 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 10:48:02 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/minicluster-data/ranger-admin/ews/webapp
Aug 06, 2026 10:48:02 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 06, 2026 10:48:02 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 06, 2026 10:48:03 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/minicluster-data/ranger-admin/ews/webapp].
Aug 06, 2026 10:48:03 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260806 22:48:04.907411 6305 mini_ranger.cc:160] Time spent starting Ranger: real 11.045s user 0.000s sys 0.014s
Aug 06, 2026 10:48:26 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 06, 2026 10:48:26 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 06, 2026 10:48:26 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 06, 2026 10:48:26 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 06, 2026 10:48:26 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 06, 2026 10:48:26 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 06, 2026 10:48:26 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 06, 2026 10:48:26 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 06, 2026 10:48:26 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 06, 2026 10:48:26 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 06, 2026 10:48:26 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 06, 2026 10:48:26 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 06, 2026 10:48:26 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 06, 2026 10:48:26 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 06, 2026 10:48:26 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 06, 2026 10:48:26 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 06, 2026 10:48:26 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 06, 2026 10:48:28 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260806 22:48:28.723485 6305 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.6.40.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.6.40.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.6.40.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/krb5kdc/rangerkms_127.6.40.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.6.40.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/krb5kdc/rangerkms_127.6.40.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.6.40.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.6.40.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.6.40.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/krb5kdc/HTTP_127.6.40.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.6.40.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/krb5kdc/HTTP_127.6.40.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 06 22:48:28 dist-test-slave-zh2d krb5kdc[21338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056508, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260806 22:48:28.817943 6305 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260806 22:48:28.832962 6305 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260806 22:48:29.788198 6305 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260806 22:48:29.788296 6305 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/minicluster-data/ranger-kms
I20260806 22:48:29.788317 6305 mini_ranger_kms.cc:227] Using postgres at 127.6.40.84:42801
2026-08-06 22:48:30,936 [I] DB FLAVOR :POSTGRES
2026-08-06 22:48:30,936 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 22:48:30,936 [I] Checking connection
2026-08-06 22:48:30,937 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_hhfSb/build/release/bin/postgresql.jar:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.40.84:42801/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-06 22:48:31,293 [I] connection success
2026-08-06 22:48:31,293 [I] --------- Verifying Ranger DB tables ---------
2026-08-06 22:48:31,293 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-06 22:48:31,293 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_hhfSb/build/release/bin/postgresql.jar:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.40.84:42801/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-06 22:48:31,554 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-06 22:48:31,555 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 22:48:31,555 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-06 22:48:31,555 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_hhfSb/build/release/bin/postgresql.jar:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.40.84:42801/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-06 22:48:31,825 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260806 22:48:32.325047 6305 mini_ranger_kms.cc:310] Created kms service
I20260806 22:48:32.549961 6305 mini_ranger_kms.cc:326] Created kudu user
I20260806 22:48:32.600135 6305 mini_ranger_kms.cc:343] Created rangerkms user
Aug 06 22:48:32 dist-test-slave-zh2d krb5kdc[21338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056508, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.6.40.84@KRBTEST.COM
I20260806 22:48:33.022421 6305 mini_ranger_kms.cc:384] Added ranger policy
I20260806 22:48:33.022567 6305 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/minicluster-data/ranger-kms:/tmp/dist-test-task_hhfSb/build/release/bin/postgresql.jar:/tmp/dist-test-task_hhfSb/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task_hhfSb/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260806 22:48:33.022593 6305 mini_ranger_kms.cc:240] Using host: 127.6.40.84
I20260806 22:48:33.027482 6305 mini_ranger_kms.cc:276] Ranger KMS PID: 21910
I20260806 22:48:33.027604 6305 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.6.40.84:46015
Aug 06, 2026 10:48:34 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 10:48:34 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 10:48:34 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 06, 2026 10:48:34 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 06, 2026 10:48:34 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/minicluster-data/ranger-kms/ews/webapp].
Aug 06, 2026 10:48:34 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260806 22:48:35.683943 6305 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.866s user 0.024s sys 0.846s
I20260806 22:48:35.684067 6305 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260806 22:48:35.684163 6305 mini_ranger_kms.cc:401] 127.6.40.84:46015/kms/v1/keys
Aug 06 22:48:35 dist-test-slave-zh2d krb5kdc[21338](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786056515, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.6.40.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 06 22:48:36 dist-test-slave-zh2d krb5kdc[21338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056515, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.6.40.84@KRBTEST.COM for HTTP/127.6.40.84@KRBTEST.COM
Aug 06 22:48:36 dist-test-slave-zh2d krb5kdc[21338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056515, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.6.40.84@KRBTEST.COM for HTTP/127.6.40.84@KRBTEST.COM
Aug 06 22:48:39 dist-test-slave-zh2d krb5kdc[21338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056519, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.6.40.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.40.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.40.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab.
Entry for principal kudu/127.6.40.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab.
I20260806 22:48:39.475792 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.126:37861
--webserver_interface=127.6.40.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab
--principal=kudu/127.6.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:34449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.40.126:37861
--ranger_config_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.40.84:46015/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:48:39.599854 21988 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:48:39.600103 21988 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:48:39.600129 21988 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:48:39.601612 21988 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:48:39.601665 21988 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:48:39.601681 21988 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:48:39.601696 21988 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:48:39.601709 21988 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:48:39.603828 21988 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:34449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/minicluster-data/master-0/wal
--ranger_kms_url=127.6.40.84:46015/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.6.40.126:37861
--ranger_config_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab
--principal=kudu/127.6.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.126:37861
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21988
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:48:39.604143 21988 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:48:39.604432 21988 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:48:39.612068 21997 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:48:39.612112 21998 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:48:39.612114 21988 server_base.cc:1034] running on GCE node
W20260806 22:48:39.612347 22001 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:48:39.612656 21988 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:48:39.613059 21988 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:48:39.614224 21988 hybrid_clock.cc:648] HybridClock initialized: now 1786056519614201 us; error 46 us; skew 500 ppm
Aug 06 22:48:39 dist-test-slave-zh2d krb5kdc[21338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056519, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:48:39.617484 21988 init.cc:377] Logged in from keytab as kudu/127.6.40.126@KRBTEST.COM (short username kudu)
I20260806 22:48:39.618072 21988 webserver.cc:460] Webserver started at http://127.6.40.126:38351/ using document root <none> and password file <none>
I20260806 22:48:39.618363 21988 fs_manager.cc:359] Metadata directory not provided
I20260806 22:48:39.618425 21988 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:48:39.618572 21988 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 22:48:39 dist-test-slave-zh2d krb5kdc[21338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056519, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.126@KRBTEST.COM for HTTP/127.6.40.84@KRBTEST.COM
I20260806 22:48:39.696506 21988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/minicluster-data/master-0/data/instance:
uuid: "f8293e32825848d8912a3a1b422517b8"
format_stamp: "Formatted at 2026-08-06 22:48:39 on dist-test-slave-zh2d"
server_key: "d27a62fd1c2314afa2a7de0efcb948a9"
server_key_iv: "a819bf24bf889f13aeac53c891581538"
server_key_version: "kuduclusterkey@0"
I20260806 22:48:39.697186 21988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/minicluster-data/master-0/wal/instance:
uuid: "f8293e32825848d8912a3a1b422517b8"
format_stamp: "Formatted at 2026-08-06 22:48:39 on dist-test-slave-zh2d"
server_key: "d27a62fd1c2314afa2a7de0efcb948a9"
server_key_iv: "a819bf24bf889f13aeac53c891581538"
server_key_version: "kuduclusterkey@0"
I20260806 22:48:39.699298 21988 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260806 22:48:39.719019 22008 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:48:39.719306 21988 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 22:48:39.719445 21988 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/minicluster-data/master-0/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/minicluster-data/master-0/wal
uuid: "f8293e32825848d8912a3a1b422517b8"
format_stamp: "Formatted at 2026-08-06 22:48:39 on dist-test-slave-zh2d"
server_key: "d27a62fd1c2314afa2a7de0efcb948a9"
server_key_iv: "a819bf24bf889f13aeac53c891581538"
server_key_version: "kuduclusterkey@0"
I20260806 22:48:39.719564 21988 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:48:39.733212 21988 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:48:39.735433 21988 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:48:39.735703 21988 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:48:39.745937 21988 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.126:37861
I20260806 22:48:39.746109 22077 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.126:37861 every 8 connection(s)
I20260806 22:48:39.746447 21988 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
I20260806 22:48:39.747499 22078 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:48:39.750780 22078 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8293e32825848d8912a3a1b422517b8: Bootstrap starting.
I20260806 22:48:39.751657 22078 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f8293e32825848d8912a3a1b422517b8: Neither blocks nor log segments found. Creating new log.
I20260806 22:48:39.752197 22078 log.cc:824] T 00000000000000000000000000000000 P f8293e32825848d8912a3a1b422517b8: Log is configured to *not* fsync() on all Append() calls
I20260806 22:48:39.753269 22078 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8293e32825848d8912a3a1b422517b8: No bootstrap required, opened a new log
I20260806 22:48:39.754925 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 21988
I20260806 22:48:39.755050 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/minicluster-data/master-0/wal/instance
I20260806 22:48:39.755285 6305 external_mini_cluster.cc:1428] Setting key f85048d736093e85888df424d6936283
I20260806 22:48:39.756898 22078 raft_consensus.cc:348] T 00000000000000000000000000000000 P f8293e32825848d8912a3a1b422517b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8293e32825848d8912a3a1b422517b8" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 37861 } }
I20260806 22:48:39.757087 22078 raft_consensus.cc:374] T 00000000000000000000000000000000 P f8293e32825848d8912a3a1b422517b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:48:39.757157 22078 raft_consensus.cc:729] T 00000000000000000000000000000000 P f8293e32825848d8912a3a1b422517b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8293e32825848d8912a3a1b422517b8, State: Initialized, Role: FOLLOWER
I20260806 22:48:39.757370 22078 consensus_queue.cc:260] T 00000000000000000000000000000000 P f8293e32825848d8912a3a1b422517b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8293e32825848d8912a3a1b422517b8" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 37861 } }
I20260806 22:48:39.757484 22078 raft_consensus.cc:388] T 00000000000000000000000000000000 P f8293e32825848d8912a3a1b422517b8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:48:39.757524 22078 raft_consensus.cc:482] T 00000000000000000000000000000000 P f8293e32825848d8912a3a1b422517b8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:48:39.757570 22078 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f8293e32825848d8912a3a1b422517b8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:48:39.758548 22078 raft_consensus.cc:504] T 00000000000000000000000000000000 P f8293e32825848d8912a3a1b422517b8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8293e32825848d8912a3a1b422517b8" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 37861 } }
I20260806 22:48:39.758682 22078 leader_election.cc:302] T 00000000000000000000000000000000 P f8293e32825848d8912a3a1b422517b8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f8293e32825848d8912a3a1b422517b8; no voters:
I20260806 22:48:39.758885 22078 leader_election.cc:288] T 00000000000000000000000000000000 P f8293e32825848d8912a3a1b422517b8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:48:39.758937 22086 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f8293e32825848d8912a3a1b422517b8 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:48:39.759163 22086 raft_consensus.cc:686] T 00000000000000000000000000000000 P f8293e32825848d8912a3a1b422517b8 [term 1 LEADER]: Becoming Leader. State: Replica: f8293e32825848d8912a3a1b422517b8, State: Running, Role: LEADER
I20260806 22:48:39.759284 22078 sys_catalog.cc:564] T 00000000000000000000000000000000 P f8293e32825848d8912a3a1b422517b8 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:48:39.759303 22086 consensus_queue.cc:237] T 00000000000000000000000000000000 P f8293e32825848d8912a3a1b422517b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8293e32825848d8912a3a1b422517b8" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 37861 } }
I20260806 22:48:39.759775 22086 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8293e32825848d8912a3a1b422517b8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f8293e32825848d8912a3a1b422517b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8293e32825848d8912a3a1b422517b8" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 37861 } } }
I20260806 22:48:39.759958 22086 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8293e32825848d8912a3a1b422517b8 [sys.catalog]: This master's current role is: LEADER
I20260806 22:48:39.760387 22088 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8293e32825848d8912a3a1b422517b8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f8293e32825848d8912a3a1b422517b8. Latest consensus state: current_term: 1 leader_uuid: "f8293e32825848d8912a3a1b422517b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8293e32825848d8912a3a1b422517b8" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 37861 } } }
I20260806 22:48:39.760571 22088 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8293e32825848d8912a3a1b422517b8 [sys.catalog]: This master's current role is: LEADER
I20260806 22:48:39.761632 22094 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:48:39.761655 22078 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
Aug 06 22:48:39 dist-test-slave-zh2d krb5kdc[21338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056519, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
I20260806 22:48:39.767748 22094 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:48:39.775493 22085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:39.755929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55146 (local address 127.6.40.126:37861)
0806 22:48:39.756216 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0806 22:48:39.756227 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:48:39.756273 (+ 46us) server_negotiation.cc:407] Connection header received
0806 22:48:39.756919 (+ 646us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:48:39.756932 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:48:39.757035 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:48:39.757225 (+ 190us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:48:39.758260 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:48:39.760163 (+ 1903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:48:39.761454 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:48:39.762245 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:48:39.762400 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:48:39.766481 (+ 4081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:48:39.766537 (+ 56us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:48:39.766538 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:48:39.766767 (+ 229us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:48:39.772374 (+ 5607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:48:39.772920 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:48:39.772924 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:48:39.772925 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:48:39.773179 (+ 254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:48:39.773441 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:48:39.773443 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:48:39.773443 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:48:39.774082 (+ 639us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:48:39.774322 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0806 22:48:39.774430 (+ 108us) server_negotiation.cc:299] Negotiation successful
0806 22:48:39.775053 (+ 623us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":223,"thread_start_us":134,"threads_started":1}
I20260806 22:48:39.780439 22094 catalog_manager.cc:1347] Generated new cluster ID: bcc6e60c66434fe781cb351a42db5112
I20260806 22:48:39.780530 22094 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:48:39.807641 22094 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:48:39.808585 22094 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:48:39.827461 22094 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f8293e32825848d8912a3a1b422517b8: Generated new TSK 0
I20260806 22:48:39.827852 22094 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.40.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.40.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.40.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.65.keytab.
Entry for principal kudu/127.6.40.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.65.keytab.
I20260806 22:48:40.756600 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.40.84:46015/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.65:0
--local_ip_for_outbound_sockets=127.6.40.65
--webserver_interface=127.6.40.65
--webserver_port=0
--tserver_master_addrs=127.6.40.126:37861
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.65.keytab
--principal=kudu/127.6.40.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:34449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:48:40.914616 22133 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:48:40.914882 22133 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:48:40.914925 22133 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:48:40.916298 22133 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:48:40.916472 22133 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:48:40.916601 22133 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.65
I20260806 22:48:40.918606 22133 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:34449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.6.40.84:46015/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.65.keytab
--principal=kudu/127.6.40.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.40.65
--webserver_port=0
--tserver_master_addrs=127.6.40.126:37861
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22133
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.65
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:48:40.918926 22133 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:48:40.919203 22133 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:48:40.926151 22144 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:48:40.926265 22147 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:48:40.926393 22133 server_base.cc:1034] running on GCE node
W20260806 22:48:40.926154 22143 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:48:40.926690 22133 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:48:40.926983 22133 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:48:40.928184 22133 hybrid_clock.cc:648] HybridClock initialized: now 1786056520928155 us; error 34 us; skew 500 ppm
Aug 06 22:48:40 dist-test-slave-zh2d krb5kdc[21338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056520, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:48:40.932089 22133 init.cc:377] Logged in from keytab as kudu/127.6.40.65@KRBTEST.COM (short username kudu)
I20260806 22:48:40.932749 22133 webserver.cc:460] Webserver started at http://127.6.40.65:40835/ using document root <none> and password file <none>
I20260806 22:48:40.933125 22133 fs_manager.cc:359] Metadata directory not provided
I20260806 22:48:40.933199 22133 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:48:40.933327 22133 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 22:48:40 dist-test-slave-zh2d krb5kdc[21338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056520, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.65@KRBTEST.COM for HTTP/127.6.40.84@KRBTEST.COM
I20260806 22:48:40.951738 22133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/minicluster-data/ts-0/data/instance:
uuid: "c7fdc2aea9e147c1a3ed5d00a2df0c86"
format_stamp: "Formatted at 2026-08-06 22:48:40 on dist-test-slave-zh2d"
server_key: "b3fbe66ca1b28810df423ef4b70e8bc2"
server_key_iv: "86350212f5e58d2c9dcea3c7dffd5a9f"
server_key_version: "kuduclusterkey@0"
I20260806 22:48:40.952282 22133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/minicluster-data/ts-0/wal/instance:
uuid: "c7fdc2aea9e147c1a3ed5d00a2df0c86"
format_stamp: "Formatted at 2026-08-06 22:48:40 on dist-test-slave-zh2d"
server_key: "b3fbe66ca1b28810df423ef4b70e8bc2"
server_key_iv: "86350212f5e58d2c9dcea3c7dffd5a9f"
server_key_version: "kuduclusterkey@0"
I20260806 22:48:40.955646 22133 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260806 22:48:40.970415 22159 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:48:40.970683 22133 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:48:40.970806 22133 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/minicluster-data/ts-0/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/minicluster-data/ts-0/wal
uuid: "c7fdc2aea9e147c1a3ed5d00a2df0c86"
format_stamp: "Formatted at 2026-08-06 22:48:40 on dist-test-slave-zh2d"
server_key: "b3fbe66ca1b28810df423ef4b70e8bc2"
server_key_iv: "86350212f5e58d2c9dcea3c7dffd5a9f"
server_key_version: "kuduclusterkey@0"
I20260806 22:48:40.970919 22133 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:48:40.986449 22133 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:48:40.989101 22133 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:48:40.989296 22133 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:48:40.989645 22133 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:48:40.990054 22133 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:48:40.990113 22133 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:48:40.990155 22133 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:48:40.990185 22133 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:48:41.011445 22133 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.65:34345
I20260806 22:48:41.011618 22319 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.65:34345 every 8 connection(s)
I20260806 22:48:41.012068 22133 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
I20260806 22:48:41.015221 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 22133
I20260806 22:48:41.015336 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/minicluster-data/ts-0/wal/instance
I20260806 22:48:41.015574 6305 external_mini_cluster.cc:1428] Setting key 99d1cc468b98a23af56814de9d24a1e8
Aug 06 22:48:41 dist-test-slave-zh2d krb5kdc[21338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056520, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.65@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
I20260806 22:48:41.028169 22323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:41.013399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:37861 (local address 127.6.40.65:47081)
0806 22:48:41.013875 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0806 22:48:41.013911 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 22:48:41.014832 (+ 921us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:48:41.015144 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:48:41.015148 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:48:41.015546 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:48:41.015986 (+ 440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:48:41.015993 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:48:41.017078 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:48:41.017080 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:48:41.018714 (+ 1634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:48:41.018719 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:48:41.019503 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:48:41.019504 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:48:41.019687 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:48:41.020535 (+ 848us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:48:41.020559 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:48:41.022842 (+ 2283us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:48:41.026065 (+ 3223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:48:41.026075 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:48:41.026078 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:48:41.026426 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:48:41.026703 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:48:41.026704 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:48:41.026704 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:48:41.026887 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:48:41.027282 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:48:41.027290 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:48:41.027533 (+ 243us) client_negotiation.cc:769] Sending connection context
0806 22:48:41.027662 (+ 129us) client_negotiation.cc:240] Negotiation successful
0806 22:48:41.027868 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":234,"threads_started":1}
WARNING: no policy specified for kudu/127.6.40.66@KRBTEST.COM; defaulting to no policy
I20260806 22:48:41.028955 22322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:41.013466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.65:47081 (local address 127.6.40.126:37861)
0806 22:48:41.013683 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0806 22:48:41.013686 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:48:41.014052 (+ 366us) server_negotiation.cc:407] Connection header received
0806 22:48:41.014981 (+ 929us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:48:41.014983 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:48:41.015001 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:48:41.015064 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:48:41.016116 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:48:41.016960 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:48:41.018904 (+ 1944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:48:41.019315 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:48:41.019413 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:48:41.022963 (+ 3550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:48:41.022965 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:48:41.022965 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:48:41.022974 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:48:41.025939 (+ 2965us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:48:41.026520 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:48:41.026522 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:48:41.026522 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:48:41.026594 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:48:41.026998 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:48:41.026999 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:48:41.027000 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:48:41.027176 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:48:41.027249 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 22:48:41.028702 (+ 1453us) server_negotiation.cc:299] Negotiation successful
0806 22:48:41.028800 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":93,"threads_started":1}
I20260806 22:48:41.029701 22320 heartbeater.cc:347] Connected to a master server at 127.6.40.126:37861
I20260806 22:48:41.029820 22320 heartbeater.cc:464] Registering TS with master...
I20260806 22:48:41.030051 22320 heartbeater.cc:511] Master 127.6.40.126:37861 requested a full tablet report, sending...
I20260806 22:48:41.030756 22029 ts_manager.cc:194] Registered new tserver with Master: c7fdc2aea9e147c1a3ed5d00a2df0c86 (127.6.40.65:34345)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.40.66@KRBTEST.COM" created.
I20260806 22:48:41.031682 22029 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.40.65@KRBTEST.COM'} at 127.6.40.65:47081
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.40.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.66.keytab.
Entry for principal kudu/127.6.40.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.66.keytab.
I20260806 22:48:41.051715 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.40.84:46015/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.66:0
--local_ip_for_outbound_sockets=127.6.40.66
--webserver_interface=127.6.40.66
--webserver_port=0
--tserver_master_addrs=127.6.40.126:37861
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.66.keytab
--principal=kudu/127.6.40.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:34449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:48:41.210215 22327 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:48:41.210523 22327 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:48:41.210566 22327 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:48:41.212376 22327 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:48:41.212442 22327 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:48:41.212488 22327 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.66
I20260806 22:48:41.214895 22327 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:34449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.6.40.84:46015/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.66.keytab
--principal=kudu/127.6.40.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.40.66
--webserver_port=0
--tserver_master_addrs=127.6.40.126:37861
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22327
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.66
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:48:41.215160 22327 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:48:41.215555 22327 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:48:41.225811 22346 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:48:41.225903 22339 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:48:41.226135 22342 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:48:41.227440 22327 server_base.cc:1034] running on GCE node
I20260806 22:48:41.227720 22327 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:48:41.228083 22327 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:48:41.229283 22327 hybrid_clock.cc:648] HybridClock initialized: now 1786056521229253 us; error 44 us; skew 500 ppm
Aug 06 22:48:41 dist-test-slave-zh2d krb5kdc[21338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056521, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:48:41.233518 22327 init.cc:377] Logged in from keytab as kudu/127.6.40.66@KRBTEST.COM (short username kudu)
I20260806 22:48:41.234297 22327 webserver.cc:460] Webserver started at http://127.6.40.66:37959/ using document root <none> and password file <none>
I20260806 22:48:41.234611 22327 fs_manager.cc:359] Metadata directory not provided
I20260806 22:48:41.234674 22327 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:48:41.234812 22327 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 22:48:41 dist-test-slave-zh2d krb5kdc[21338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056521, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.66@KRBTEST.COM for HTTP/127.6.40.84@KRBTEST.COM
Aug 06 22:48:41 dist-test-slave-zh2d krb5kdc[21338](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786056521, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:48:41.264626 22327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/minicluster-data/ts-1/data/instance:
uuid: "d2af6fdc8a424f808895e2bee1f3a818"
format_stamp: "Formatted at 2026-08-06 22:48:41 on dist-test-slave-zh2d"
server_key: "c145f6fe6f2ce1412bc68cb9d1996981"
server_key_iv: "05c8ed21deb98c2ef2a2937dbe772217"
server_key_version: "kuduclusterkey@0"
I20260806 22:48:41.265306 22327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/minicluster-data/ts-1/wal/instance:
uuid: "d2af6fdc8a424f808895e2bee1f3a818"
format_stamp: "Formatted at 2026-08-06 22:48:41 on dist-test-slave-zh2d"
server_key: "c145f6fe6f2ce1412bc68cb9d1996981"
server_key_iv: "05c8ed21deb98c2ef2a2937dbe772217"
server_key_version: "kuduclusterkey@0"
I20260806 22:48:41.281037 22327 fs_manager.cc:691] Time spent creating directory manager: real 0.015s user 0.002s sys 0.000s
I20260806 22:48:41.299621 22356 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:48:41.300187 22327 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:48:41.300352 22327 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/minicluster-data/ts-1/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/minicluster-data/ts-1/wal
uuid: "d2af6fdc8a424f808895e2bee1f3a818"
format_stamp: "Formatted at 2026-08-06 22:48:41 on dist-test-slave-zh2d"
server_key: "c145f6fe6f2ce1412bc68cb9d1996981"
server_key_iv: "05c8ed21deb98c2ef2a2937dbe772217"
server_key_version: "kuduclusterkey@0"
I20260806 22:48:41.300547 22327 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:48:41.317685 22327 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:48:41.320024 22327 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:48:41.320232 22327 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:48:41.320636 22327 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:48:41.321220 22327 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:48:41.321272 22327 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:48:41.321311 22327 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:48:41.321342 22327 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:48:41.340150 22327 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.66:38353
I20260806 22:48:41.340670 22327 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
I20260806 22:48:41.341620 22518 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.66:38353 every 8 connection(s)
Aug 06 22:48:41 dist-test-slave-zh2d krb5kdc[21338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056521, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.66@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
I20260806 22:48:41.350852 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 22327
I20260806 22:48:41.350961 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/minicluster-data/ts-1/wal/instance
I20260806 22:48:41.351112 6305 external_mini_cluster.cc:1428] Setting key eb6fdcd44506cb6b01eca693fbb343ab
I20260806 22:48:41.354786 22322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:41.342885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.66:55775 (local address 127.6.40.126:37861)
0806 22:48:41.343072 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0806 22:48:41.343075 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:48:41.343157 (+ 82us) server_negotiation.cc:407] Connection header received
0806 22:48:41.344584 (+ 1427us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:48:41.344585 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:48:41.344598 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:48:41.344661 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:48:41.345726 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:48:41.346257 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:48:41.347709 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:48:41.347975 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:48:41.348058 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:48:41.350426 (+ 2368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:48:41.350427 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:48:41.350428 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:48:41.350436 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:48:41.352575 (+ 2139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:48:41.353228 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:48:41.353230 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:48:41.353230 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:48:41.353330 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:48:41.353708 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:48:41.353709 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:48:41.353710 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:48:41.353997 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:48:41.354059 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 22:48:41.354541 (+ 482us) server_negotiation.cc:299] Negotiation successful
0806 22:48:41.354647 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158}
I20260806 22:48:41.355430 22521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:41.342657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:37861 (local address 127.6.40.66:55775)
0806 22:48:41.343003 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0806 22:48:41.343049 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 22:48:41.344450 (+ 1401us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:48:41.344736 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:48:41.344740 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:48:41.345322 (+ 582us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:48:41.345616 (+ 294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:48:41.345622 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:48:41.346364 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:48:41.346365 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:48:41.347611 (+ 1246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:48:41.347615 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:48:41.348093 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:48:41.348094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:48:41.348240 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:48:41.348925 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:48:41.348934 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:48:41.350315 (+ 1381us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:48:41.352665 (+ 2350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:48:41.352673 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:48:41.352675 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:48:41.353132 (+ 457us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:48:41.353411 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:48:41.353411 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:48:41.353412 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:48:41.353577 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:48:41.354098 (+ 521us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:48:41.354106 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:48:41.354370 (+ 264us) client_negotiation.cc:769] Sending connection context
0806 22:48:41.354892 (+ 522us) client_negotiation.cc:240] Negotiation successful
0806 22:48:41.355081 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":218,"threads_started":1}
I20260806 22:48:41.356359 22519 heartbeater.cc:347] Connected to a master server at 127.6.40.126:37861
I20260806 22:48:41.356473 22519 heartbeater.cc:464] Registering TS with master...
I20260806 22:48:41.356709 22519 heartbeater.cc:511] Master 127.6.40.126:37861 requested a full tablet report, sending...
I20260806 22:48:41.357141 22029 ts_manager.cc:194] Registered new tserver with Master: d2af6fdc8a424f808895e2bee1f3a818 (127.6.40.66:38353)
I20260806 22:48:41.357664 22029 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.40.66@KRBTEST.COM'} at 127.6.40.66:55775
WARNING: no policy specified for kudu/127.6.40.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.40.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.40.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.67.keytab.
Entry for principal kudu/127.6.40.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.67.keytab.
I20260806 22:48:41.399730 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.40.84:46015/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.67:0
--local_ip_for_outbound_sockets=127.6.40.67
--webserver_interface=127.6.40.67
--webserver_port=0
--tserver_master_addrs=127.6.40.126:37861
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.67.keytab
--principal=kudu/127.6.40.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:34449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:48:41.596637 22529 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:48:41.597084 22529 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:48:41.597219 22529 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:48:41.599519 22529 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:48:41.599596 22529 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:48:41.599646 22529 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.67
I20260806 22:48:41.602589 22529 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:34449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.6.40.84:46015/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.67.keytab
--principal=kudu/127.6.40.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.40.67
--webserver_port=0
--tserver_master_addrs=127.6.40.126:37861
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22529
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.67
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:48:41.602910 22529 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:48:41.603281 22529 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:48:41.614928 22536 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:48:41.615068 22541 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:48:41.616150 22539 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:48:41.616310 22529 server_base.cc:1034] running on GCE node
I20260806 22:48:41.616554 22529 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:48:41.616912 22529 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:48:41.624369 22529 hybrid_clock.cc:648] HybridClock initialized: now 1786056521624326 us; error 49 us; skew 500 ppm
Aug 06 22:48:41 dist-test-slave-zh2d krb5kdc[21338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056521, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:48:41.629024 22529 init.cc:377] Logged in from keytab as kudu/127.6.40.67@KRBTEST.COM (short username kudu)
I20260806 22:48:41.629689 22529 webserver.cc:460] Webserver started at http://127.6.40.67:44869/ using document root <none> and password file <none>
I20260806 22:48:41.630060 22529 fs_manager.cc:359] Metadata directory not provided
I20260806 22:48:41.630194 22529 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:48:41.630355 22529 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 22:48:41 dist-test-slave-zh2d krb5kdc[21338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056521, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.67@KRBTEST.COM for HTTP/127.6.40.84@KRBTEST.COM
I20260806 22:48:41.652815 22529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/minicluster-data/ts-2/data/instance:
uuid: "68be8534f09b4cd6b7ac9fe06b73a7e1"
format_stamp: "Formatted at 2026-08-06 22:48:41 on dist-test-slave-zh2d"
server_key: "bcfd867b12d8ca51b7bd1f208f068884"
server_key_iv: "6c6c97364c6c2664b2264e8a67ff6dc3"
server_key_version: "kuduclusterkey@0"
I20260806 22:48:41.653434 22529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/minicluster-data/ts-2/wal/instance:
uuid: "68be8534f09b4cd6b7ac9fe06b73a7e1"
format_stamp: "Formatted at 2026-08-06 22:48:41 on dist-test-slave-zh2d"
server_key: "bcfd867b12d8ca51b7bd1f208f068884"
server_key_iv: "6c6c97364c6c2664b2264e8a67ff6dc3"
server_key_version: "kuduclusterkey@0"
I20260806 22:48:41.658473 22529 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.002s
I20260806 22:48:41.678056 22549 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:48:41.678277 22529 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 22:48:41.678397 22529 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/minicluster-data/ts-2/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/minicluster-data/ts-2/wal
uuid: "68be8534f09b4cd6b7ac9fe06b73a7e1"
format_stamp: "Formatted at 2026-08-06 22:48:41 on dist-test-slave-zh2d"
server_key: "bcfd867b12d8ca51b7bd1f208f068884"
server_key_iv: "6c6c97364c6c2664b2264e8a67ff6dc3"
server_key_version: "kuduclusterkey@0"
I20260806 22:48:41.678547 22529 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:48:41.695003 22529 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:48:41.697582 22529 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:48:41.697832 22529 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:48:41.698263 22529 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:48:41.698747 22529 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:48:41.698807 22529 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:48:41.698861 22529 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:48:41.698894 22529 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:48:41.716761 22529 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.67:40325
I20260806 22:48:41.717010 22709 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.67:40325 every 8 connection(s)
I20260806 22:48:41.717374 22529 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
Aug 06 22:48:41 dist-test-slave-zh2d krb5kdc[21338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056521, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.67@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
I20260806 22:48:41.727510 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 22529
I20260806 22:48:41.727620 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/minicluster-data/ts-2/wal/instance
I20260806 22:48:41.727960 6305 external_mini_cluster.cc:1428] Setting key 96d7ac5138f2e07b9d97350aa52ca2ae
I20260806 22:48:41.729887 22322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:41.718834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.67:39127 (local address 127.6.40.126:37861)
0806 22:48:41.718907 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0806 22:48:41.718909 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:48:41.719031 (+ 122us) server_negotiation.cc:407] Connection header received
0806 22:48:41.719946 (+ 915us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:48:41.719948 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:48:41.719962 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:48:41.720022 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:48:41.720971 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:48:41.721506 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:48:41.722869 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:48:41.723139 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:48:41.723221 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:48:41.726040 (+ 2819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:48:41.726041 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:48:41.726042 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:48:41.726050 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:48:41.727820 (+ 1770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:48:41.728693 (+ 873us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:48:41.728694 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:48:41.728695 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:48:41.728761 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:48:41.729099 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:48:41.729100 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:48:41.729100 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:48:41.729227 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:48:41.729308 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 22:48:41.729702 (+ 394us) server_negotiation.cc:299] Negotiation successful
0806 22:48:41.729792 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 22:48:41.730168 22714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:41.718493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:37861 (local address 127.6.40.67:39127)
0806 22:48:41.718775 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0806 22:48:41.718825 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 22:48:41.719820 (+ 995us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:48:41.720105 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:48:41.720109 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:48:41.720498 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:48:41.720832 (+ 334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:48:41.720886 (+ 54us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:48:41.721624 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:48:41.721625 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:48:41.722774 (+ 1149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:48:41.722777 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:48:41.723233 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:48:41.723234 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:48:41.723449 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:48:41.724159 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:48:41.724168 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:48:41.725929 (+ 1761us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:48:41.728145 (+ 2216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:48:41.728155 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:48:41.728158 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:48:41.728605 (+ 447us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:48:41.728869 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:48:41.728870 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:48:41.728871 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:48:41.729019 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:48:41.729310 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:48:41.729319 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:48:41.729543 (+ 224us) client_negotiation.cc:769] Sending connection context
0806 22:48:41.729660 (+ 117us) client_negotiation.cc:240] Negotiation successful
0806 22:48:41.729833 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":135,"threads_started":1}
I20260806 22:48:41.730830 22711 heartbeater.cc:347] Connected to a master server at 127.6.40.126:37861
I20260806 22:48:41.730952 22711 heartbeater.cc:464] Registering TS with master...
I20260806 22:48:41.731148 22711 heartbeater.cc:511] Master 127.6.40.126:37861 requested a full tablet report, sending...
I20260806 22:48:41.731559 22029 ts_manager.cc:194] Registered new tserver with Master: 68be8534f09b4cd6b7ac9fe06b73a7e1 (127.6.40.67:40325)
I20260806 22:48:41.731946 22029 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.40.67@KRBTEST.COM'} at 127.6.40.67:39127
I20260806 22:48:41.739655 6305 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:48:41.749682 22322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:41.741354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55150 (local address 127.6.40.126:37861)
0806 22:48:41.741399 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 22:48:41.741401 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:48:41.741767 (+ 366us) server_negotiation.cc:407] Connection header received
0806 22:48:41.741855 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:48:41.741856 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:48:41.741868 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:48:41.741918 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:48:41.742614 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:48:41.743303 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:48:41.744388 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:48:41.744669 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:48:41.744716 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:48:41.746537 (+ 1821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:48:41.746538 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:48:41.746539 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:48:41.746546 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:48:41.748327 (+ 1781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:48:41.748763 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:48:41.748764 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:48:41.748765 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:48:41.748824 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:48:41.749076 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:48:41.749077 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:48:41.749077 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:48:41.749202 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:48:41.749276 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 22:48:41.749510 (+ 234us) server_negotiation.cc:299] Negotiation successful
0806 22:48:41.749590 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 22:48:41.754334 22029 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55150:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:48:41.757230 22029 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:48:41.779228 22730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:41.766718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.66:38353 (local address 127.0.0.1:51800)
0806 22:48:41.773352 (+ 6634us) negotiation.cc:107] Waiting for socket to connect
0806 22:48:41.773362 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:48:41.773560 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:48:41.774123 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:48:41.774127 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:48:41.774130 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:48:41.774216 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:48:41.774218 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:48:41.775851 (+ 1633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:48:41.775853 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:48:41.777327 (+ 1474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:48:41.777331 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:48:41.778889 (+ 1558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:48:41.778889 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:48:41.778983 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:48:41.778999 (+ 16us) client_negotiation.cc:769] Sending connection context
0806 22:48:41.779073 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 22:48:41.779129 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6611,"thread_start_us":169,"threads_started":1}
I20260806 22:48:41.779525 22733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:41.767514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51800 (local address 127.6.40.66:38353)
0806 22:48:41.767830 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0806 22:48:41.767835 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:48:41.773632 (+ 5797us) server_negotiation.cc:407] Connection header received
0806 22:48:41.773726 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:48:41.773730 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:48:41.773750 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:48:41.773896 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:48:41.774315 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:48:41.775716 (+ 1401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:48:41.777521 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:48:41.778394 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:48:41.778466 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:48:41.778601 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0806 22:48:41.779376 (+ 775us) server_negotiation.cc:299] Negotiation successful
0806 22:48:41.779422 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":253,"thread_start_us":148,"threads_started":1}
I20260806 22:48:41.780709 22426 tablet_service.cc:1467] Processing CreateTablet for tablet 22ea4d56c866498b87737a4c2c8a9563 (DEFAULT_TABLE table=test-table [id=445ef27eea4a4afeb519cda5e5901771]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:48:41.781241 22426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 22ea4d56c866498b87737a4c2c8a9563. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:48:41.784632 22727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:41.766701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.67:40325 (local address 127.0.0.1:54746)
0806 22:48:41.773014 (+ 6313us) negotiation.cc:107] Waiting for socket to connect
0806 22:48:41.773092 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0806 22:48:41.773197 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:48:41.780120 (+ 6923us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:48:41.780121 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:48:41.780123 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:48:41.780186 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:48:41.780188 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:48:41.781845 (+ 1657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:48:41.781846 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:48:41.783018 (+ 1172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:48:41.783021 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:48:41.784386 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:48:41.784387 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:48:41.784453 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:48:41.784459 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 22:48:41.784492 (+ 33us) client_negotiation.cc:240] Negotiation successful
0806 22:48:41.784541 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6264,"thread_start_us":187,"threads_started":1}
I20260806 22:48:41.785099 22732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:41.767194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54746 (local address 127.6.40.67:40325)
0806 22:48:41.779750 (+ 12556us) server_negotiation.cc:206] Beginning negotiation
0806 22:48:41.779756 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:48:41.779768 (+ 12us) server_negotiation.cc:407] Connection header received
0806 22:48:41.779815 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:48:41.779818 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:48:41.779834 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:48:41.780269 (+ 435us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:48:41.780387 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:48:41.781704 (+ 1317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:48:41.783311 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:48:41.784278 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:48:41.784753 (+ 475us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:48:41.784905 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0806 22:48:41.784940 (+ 35us) server_negotiation.cc:299] Negotiation successful
0806 22:48:41.784988 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12491,"thread_start_us":150,"threads_started":1}
I20260806 22:48:41.785918 22613 tablet_service.cc:1467] Processing CreateTablet for tablet 22ea4d56c866498b87737a4c2c8a9563 (DEFAULT_TABLE table=test-table [id=445ef27eea4a4afeb519cda5e5901771]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:48:41.786366 22613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 22ea4d56c866498b87737a4c2c8a9563. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:48:41.790481 22731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:41.767236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39114 (local address 127.6.40.65:34345)
0806 22:48:41.772958 (+ 5722us) server_negotiation.cc:206] Beginning negotiation
0806 22:48:41.772963 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:48:41.778836 (+ 5873us) server_negotiation.cc:407] Connection header received
0806 22:48:41.778890 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:48:41.778893 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:48:41.778913 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:48:41.779025 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:48:41.779278 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:48:41.780495 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:48:41.788959 (+ 8464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:48:41.789933 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:48:41.789995 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:48:41.790071 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 22:48:41.790348 (+ 277us) server_negotiation.cc:299] Negotiation successful
0806 22:48:41.790394 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5660,"thread_start_us":59,"threads_started":1}
I20260806 22:48:41.790773 22729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:41.767107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.65:34345 (local address 127.0.0.1:39114)
0806 22:48:41.778686 (+ 11579us) negotiation.cc:107] Waiting for socket to connect
0806 22:48:41.778698 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:48:41.778818 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:48:41.779120 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:48:41.779121 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:48:41.779124 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:48:41.779181 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:48:41.779183 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:48:41.780605 (+ 1422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:48:41.780606 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:48:41.781774 (+ 1168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:48:41.781777 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:48:41.790164 (+ 8387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:48:41.790165 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:48:41.790240 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:48:41.790245 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:48:41.790627 (+ 382us) client_negotiation.cc:240] Negotiation successful
0806 22:48:41.790685 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11551,"thread_start_us":96,"threads_started":1}
I20260806 22:48:41.791736 22211 tablet_service.cc:1467] Processing CreateTablet for tablet 22ea4d56c866498b87737a4c2c8a9563 (DEFAULT_TABLE table=test-table [id=445ef27eea4a4afeb519cda5e5901771]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:48:41.792121 22211 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 22ea4d56c866498b87737a4c2c8a9563. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:48:41.793138 22734 tablet_bootstrap.cc:492] T 22ea4d56c866498b87737a4c2c8a9563 P d2af6fdc8a424f808895e2bee1f3a818: Bootstrap starting.
I20260806 22:48:41.794809 22734 tablet_bootstrap.cc:654] T 22ea4d56c866498b87737a4c2c8a9563 P d2af6fdc8a424f808895e2bee1f3a818: Neither blocks nor log segments found. Creating new log.
I20260806 22:48:41.795140 22735 tablet_bootstrap.cc:492] T 22ea4d56c866498b87737a4c2c8a9563 P 68be8534f09b4cd6b7ac9fe06b73a7e1: Bootstrap starting.
I20260806 22:48:41.796283 22736 tablet_bootstrap.cc:492] T 22ea4d56c866498b87737a4c2c8a9563 P c7fdc2aea9e147c1a3ed5d00a2df0c86: Bootstrap starting.
I20260806 22:48:41.796804 22735 tablet_bootstrap.cc:654] T 22ea4d56c866498b87737a4c2c8a9563 P 68be8534f09b4cd6b7ac9fe06b73a7e1: Neither blocks nor log segments found. Creating new log.
I20260806 22:48:41.797422 22735 log.cc:824] T 22ea4d56c866498b87737a4c2c8a9563 P 68be8534f09b4cd6b7ac9fe06b73a7e1: Log is configured to *not* fsync() on all Append() calls
I20260806 22:48:41.797930 22736 tablet_bootstrap.cc:654] T 22ea4d56c866498b87737a4c2c8a9563 P c7fdc2aea9e147c1a3ed5d00a2df0c86: Neither blocks nor log segments found. Creating new log.
I20260806 22:48:41.798094 22734 log.cc:824] T 22ea4d56c866498b87737a4c2c8a9563 P d2af6fdc8a424f808895e2bee1f3a818: Log is configured to *not* fsync() on all Append() calls
I20260806 22:48:41.798678 22736 log.cc:824] T 22ea4d56c866498b87737a4c2c8a9563 P c7fdc2aea9e147c1a3ed5d00a2df0c86: Log is configured to *not* fsync() on all Append() calls
I20260806 22:48:41.799593 22736 tablet_bootstrap.cc:492] T 22ea4d56c866498b87737a4c2c8a9563 P c7fdc2aea9e147c1a3ed5d00a2df0c86: No bootstrap required, opened a new log
I20260806 22:48:41.799719 22736 ts_tablet_manager.cc:1397] T 22ea4d56c866498b87737a4c2c8a9563 P c7fdc2aea9e147c1a3ed5d00a2df0c86: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.003s
I20260806 22:48:41.799767 22735 tablet_bootstrap.cc:492] T 22ea4d56c866498b87737a4c2c8a9563 P 68be8534f09b4cd6b7ac9fe06b73a7e1: No bootstrap required, opened a new log
I20260806 22:48:41.799921 22735 ts_tablet_manager.cc:1397] T 22ea4d56c866498b87737a4c2c8a9563 P 68be8534f09b4cd6b7ac9fe06b73a7e1: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260806 22:48:41.803236 22735 raft_consensus.cc:348] T 22ea4d56c866498b87737a4c2c8a9563 P 68be8534f09b4cd6b7ac9fe06b73a7e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68be8534f09b4cd6b7ac9fe06b73a7e1" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 40325 } } peers { permanent_uuid: "d2af6fdc8a424f808895e2bee1f3a818" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 38353 } } peers { permanent_uuid: "c7fdc2aea9e147c1a3ed5d00a2df0c86" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 34345 } }
I20260806 22:48:41.803532 22735 raft_consensus.cc:374] T 22ea4d56c866498b87737a4c2c8a9563 P 68be8534f09b4cd6b7ac9fe06b73a7e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:48:41.803615 22735 raft_consensus.cc:729] T 22ea4d56c866498b87737a4c2c8a9563 P 68be8534f09b4cd6b7ac9fe06b73a7e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68be8534f09b4cd6b7ac9fe06b73a7e1, State: Initialized, Role: FOLLOWER
I20260806 22:48:41.803812 22735 consensus_queue.cc:260] T 22ea4d56c866498b87737a4c2c8a9563 P 68be8534f09b4cd6b7ac9fe06b73a7e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68be8534f09b4cd6b7ac9fe06b73a7e1" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 40325 } } peers { permanent_uuid: "d2af6fdc8a424f808895e2bee1f3a818" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 38353 } } peers { permanent_uuid: "c7fdc2aea9e147c1a3ed5d00a2df0c86" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 34345 } }
I20260806 22:48:41.804051 22736 raft_consensus.cc:348] T 22ea4d56c866498b87737a4c2c8a9563 P c7fdc2aea9e147c1a3ed5d00a2df0c86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68be8534f09b4cd6b7ac9fe06b73a7e1" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 40325 } } peers { permanent_uuid: "d2af6fdc8a424f808895e2bee1f3a818" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 38353 } } peers { permanent_uuid: "c7fdc2aea9e147c1a3ed5d00a2df0c86" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 34345 } }
I20260806 22:48:41.804256 22736 raft_consensus.cc:374] T 22ea4d56c866498b87737a4c2c8a9563 P c7fdc2aea9e147c1a3ed5d00a2df0c86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:48:41.804337 22736 raft_consensus.cc:729] T 22ea4d56c866498b87737a4c2c8a9563 P c7fdc2aea9e147c1a3ed5d00a2df0c86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7fdc2aea9e147c1a3ed5d00a2df0c86, State: Initialized, Role: FOLLOWER
I20260806 22:48:41.804471 22735 ts_tablet_manager.cc:1428] T 22ea4d56c866498b87737a4c2c8a9563 P 68be8534f09b4cd6b7ac9fe06b73a7e1: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260806 22:48:41.804544 22736 consensus_queue.cc:260] T 22ea4d56c866498b87737a4c2c8a9563 P c7fdc2aea9e147c1a3ed5d00a2df0c86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68be8534f09b4cd6b7ac9fe06b73a7e1" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 40325 } } peers { permanent_uuid: "d2af6fdc8a424f808895e2bee1f3a818" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 38353 } } peers { permanent_uuid: "c7fdc2aea9e147c1a3ed5d00a2df0c86" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 34345 } }
I20260806 22:48:41.804638 22711 heartbeater.cc:503] Master 127.6.40.126:37861 was elected leader, sending a full tablet report...
I20260806 22:48:41.805099 22736 ts_tablet_manager.cc:1428] T 22ea4d56c866498b87737a4c2c8a9563 P c7fdc2aea9e147c1a3ed5d00a2df0c86: Time spent starting tablet: real 0.005s user 0.004s sys 0.001s
I20260806 22:48:41.805234 22320 heartbeater.cc:503] Master 127.6.40.126:37861 was elected leader, sending a full tablet report...
I20260806 22:48:41.805580 22734 tablet_bootstrap.cc:492] T 22ea4d56c866498b87737a4c2c8a9563 P d2af6fdc8a424f808895e2bee1f3a818: No bootstrap required, opened a new log
I20260806 22:48:41.805656 22734 ts_tablet_manager.cc:1397] T 22ea4d56c866498b87737a4c2c8a9563 P d2af6fdc8a424f808895e2bee1f3a818: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.000s
I20260806 22:48:41.808763 22734 raft_consensus.cc:348] T 22ea4d56c866498b87737a4c2c8a9563 P d2af6fdc8a424f808895e2bee1f3a818 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68be8534f09b4cd6b7ac9fe06b73a7e1" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 40325 } } peers { permanent_uuid: "d2af6fdc8a424f808895e2bee1f3a818" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 38353 } } peers { permanent_uuid: "c7fdc2aea9e147c1a3ed5d00a2df0c86" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 34345 } }
I20260806 22:48:41.809012 22734 raft_consensus.cc:374] T 22ea4d56c866498b87737a4c2c8a9563 P d2af6fdc8a424f808895e2bee1f3a818 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:48:41.809094 22734 raft_consensus.cc:729] T 22ea4d56c866498b87737a4c2c8a9563 P d2af6fdc8a424f808895e2bee1f3a818 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2af6fdc8a424f808895e2bee1f3a818, State: Initialized, Role: FOLLOWER
I20260806 22:48:41.809248 22734 consensus_queue.cc:260] T 22ea4d56c866498b87737a4c2c8a9563 P d2af6fdc8a424f808895e2bee1f3a818 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68be8534f09b4cd6b7ac9fe06b73a7e1" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 40325 } } peers { permanent_uuid: "d2af6fdc8a424f808895e2bee1f3a818" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 38353 } } peers { permanent_uuid: "c7fdc2aea9e147c1a3ed5d00a2df0c86" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 34345 } }
I20260806 22:48:41.809517 22734 ts_tablet_manager.cc:1428] T 22ea4d56c866498b87737a4c2c8a9563 P d2af6fdc8a424f808895e2bee1f3a818: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260806 22:48:41.809670 22519 heartbeater.cc:503] Master 127.6.40.126:37861 was elected leader, sending a full tablet report...
W20260806 22:48:41.842924 22520 tablet.cc:2369] T 22ea4d56c866498b87737a4c2c8a9563 P d2af6fdc8a424f808895e2bee1f3a818: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 06 22:48:41 dist-test-slave-zh2d krb5kdc[21338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056521, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.126@KRBTEST.COM for HTTP/127.6.40.84@KRBTEST.COM
W20260806 22:48:41.969415 22713 tablet.cc:2369] T 22ea4d56c866498b87737a4c2c8a9563 P 68be8534f09b4cd6b7ac9fe06b73a7e1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:48:42.013839 22321 tablet.cc:2369] T 22ea4d56c866498b87737a4c2c8a9563 P c7fdc2aea9e147c1a3ed5d00a2df0c86: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:48:42.044727 22745 raft_consensus.cc:482] T 22ea4d56c866498b87737a4c2c8a9563 P 68be8534f09b4cd6b7ac9fe06b73a7e1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:48:42.044919 22745 raft_consensus.cc:504] T 22ea4d56c866498b87737a4c2c8a9563 P 68be8534f09b4cd6b7ac9fe06b73a7e1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68be8534f09b4cd6b7ac9fe06b73a7e1" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 40325 } } peers { permanent_uuid: "d2af6fdc8a424f808895e2bee1f3a818" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 38353 } } peers { permanent_uuid: "c7fdc2aea9e147c1a3ed5d00a2df0c86" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 34345 } }
I20260806 22:48:42.045553 22745 leader_election.cc:288] T 22ea4d56c866498b87737a4c2c8a9563 P 68be8534f09b4cd6b7ac9fe06b73a7e1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2af6fdc8a424f808895e2bee1f3a818 (127.6.40.66:38353), c7fdc2aea9e147c1a3ed5d00a2df0c86 (127.6.40.65:34345)
I20260806 22:48:42.049810 22714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:42.045754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.66:38353 (local address 127.6.40.67:60657)
0806 22:48:42.045872 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0806 22:48:42.045884 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:48:42.045931 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:48:42.046216 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:48:42.046217 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:48:42.046219 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:48:42.046343 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:48:42.046346 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:48:42.047293 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:48:42.047294 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:48:42.048545 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:48:42.048548 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:48:42.049486 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:48:42.049487 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:48:42.049633 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:48:42.049637 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:48:42.049689 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 22:48:42.049735 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96}
I20260806 22:48:42.050326 22733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:42.046012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.67:60657 (local address 127.6.40.66:38353)
0806 22:48:42.046059 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 22:48:42.046061 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:48:42.046070 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:48:42.046103 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:48:42.046104 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:48:42.046116 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:48:42.046161 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:48:42.046439 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:48:42.047150 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:48:42.048771 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:48:42.049319 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:48:42.049437 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:48:42.049451 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0806 22:48:42.050206 (+ 755us) server_negotiation.cc:299] Negotiation successful
0806 22:48:42.050259 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 22:48:42.050688 22441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "22ea4d56c866498b87737a4c2c8a9563" candidate_uuid: "68be8534f09b4cd6b7ac9fe06b73a7e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2af6fdc8a424f808895e2bee1f3a818" is_pre_election: true
I20260806 22:48:42.050853 22441 raft_consensus.cc:2454] T 22ea4d56c866498b87737a4c2c8a9563 P d2af6fdc8a424f808895e2bee1f3a818 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68be8534f09b4cd6b7ac9fe06b73a7e1 in term 0.
I20260806 22:48:42.050937 22731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:42.046086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.67:35557 (local address 127.6.40.65:34345)
0806 22:48:42.046129 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 22:48:42.046132 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:48:42.046371 (+ 239us) server_negotiation.cc:407] Connection header received
0806 22:48:42.046479 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:48:42.046481 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:48:42.046495 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:48:42.046524 (+ 29us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:48:42.046946 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:48:42.047761 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:48:42.049819 (+ 2058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:48:42.050387 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:48:42.050433 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:48:42.050443 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 22:48:42.050818 (+ 375us) server_negotiation.cc:299] Negotiation successful
0806 22:48:42.050866 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 22:48:42.051339 22749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:42.045776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.65:34345 (local address 127.6.40.67:35557)
0806 22:48:42.046288 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0806 22:48:42.046299 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:48:42.046402 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:48:42.046586 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:48:42.046587 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:48:42.046589 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:48:42.046854 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:48:42.046856 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:48:42.047863 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:48:42.047864 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:48:42.049704 (+ 1840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:48:42.049708 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:48:42.050575 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:48:42.050576 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:48:42.050723 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:48:42.050728 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:48:42.051052 (+ 324us) client_negotiation.cc:240] Negotiation successful
0806 22:48:42.051097 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":192,"threads_started":1}
I20260806 22:48:42.051782 22552 leader_election.cc:302] T 22ea4d56c866498b87737a4c2c8a9563 P 68be8534f09b4cd6b7ac9fe06b73a7e1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68be8534f09b4cd6b7ac9fe06b73a7e1, d2af6fdc8a424f808895e2bee1f3a818; no voters:
I20260806 22:48:42.051735 22252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "22ea4d56c866498b87737a4c2c8a9563" candidate_uuid: "68be8534f09b4cd6b7ac9fe06b73a7e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7fdc2aea9e147c1a3ed5d00a2df0c86" is_pre_election: true
I20260806 22:48:42.051918 22252 raft_consensus.cc:2454] T 22ea4d56c866498b87737a4c2c8a9563 P c7fdc2aea9e147c1a3ed5d00a2df0c86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68be8534f09b4cd6b7ac9fe06b73a7e1 in term 0.
I20260806 22:48:42.051929 22745 raft_consensus.cc:2781] T 22ea4d56c866498b87737a4c2c8a9563 P 68be8534f09b4cd6b7ac9fe06b73a7e1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:48:42.051976 22745 raft_consensus.cc:482] T 22ea4d56c866498b87737a4c2c8a9563 P 68be8534f09b4cd6b7ac9fe06b73a7e1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:48:42.052006 22745 raft_consensus.cc:3037] T 22ea4d56c866498b87737a4c2c8a9563 P 68be8534f09b4cd6b7ac9fe06b73a7e1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:48:42.053977 22745 raft_consensus.cc:504] T 22ea4d56c866498b87737a4c2c8a9563 P 68be8534f09b4cd6b7ac9fe06b73a7e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68be8534f09b4cd6b7ac9fe06b73a7e1" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 40325 } } peers { permanent_uuid: "d2af6fdc8a424f808895e2bee1f3a818" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 38353 } } peers { permanent_uuid: "c7fdc2aea9e147c1a3ed5d00a2df0c86" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 34345 } }
I20260806 22:48:42.054121 22745 leader_election.cc:288] T 22ea4d56c866498b87737a4c2c8a9563 P 68be8534f09b4cd6b7ac9fe06b73a7e1 [CANDIDATE]: Term 1 election: Requested vote from peers d2af6fdc8a424f808895e2bee1f3a818 (127.6.40.66:38353), c7fdc2aea9e147c1a3ed5d00a2df0c86 (127.6.40.65:34345)
I20260806 22:48:42.054491 22252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "22ea4d56c866498b87737a4c2c8a9563" candidate_uuid: "68be8534f09b4cd6b7ac9fe06b73a7e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7fdc2aea9e147c1a3ed5d00a2df0c86"
I20260806 22:48:42.054569 22252 raft_consensus.cc:3037] T 22ea4d56c866498b87737a4c2c8a9563 P c7fdc2aea9e147c1a3ed5d00a2df0c86 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:48:42.055841 22252 raft_consensus.cc:2454] T 22ea4d56c866498b87737a4c2c8a9563 P c7fdc2aea9e147c1a3ed5d00a2df0c86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68be8534f09b4cd6b7ac9fe06b73a7e1 in term 1.
I20260806 22:48:42.056110 22441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "22ea4d56c866498b87737a4c2c8a9563" candidate_uuid: "68be8534f09b4cd6b7ac9fe06b73a7e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2af6fdc8a424f808895e2bee1f3a818"
I20260806 22:48:42.056146 22550 leader_election.cc:302] T 22ea4d56c866498b87737a4c2c8a9563 P 68be8534f09b4cd6b7ac9fe06b73a7e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68be8534f09b4cd6b7ac9fe06b73a7e1, c7fdc2aea9e147c1a3ed5d00a2df0c86; no voters:
I20260806 22:48:42.056202 22441 raft_consensus.cc:3037] T 22ea4d56c866498b87737a4c2c8a9563 P d2af6fdc8a424f808895e2bee1f3a818 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:48:42.057001 22745 raft_consensus.cc:2781] T 22ea4d56c866498b87737a4c2c8a9563 P 68be8534f09b4cd6b7ac9fe06b73a7e1 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:48:42.057269 22745 raft_consensus.cc:686] T 22ea4d56c866498b87737a4c2c8a9563 P 68be8534f09b4cd6b7ac9fe06b73a7e1 [term 1 LEADER]: Becoming Leader. State: Replica: 68be8534f09b4cd6b7ac9fe06b73a7e1, State: Running, Role: LEADER
I20260806 22:48:42.057381 22745 consensus_queue.cc:237] T 22ea4d56c866498b87737a4c2c8a9563 P 68be8534f09b4cd6b7ac9fe06b73a7e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68be8534f09b4cd6b7ac9fe06b73a7e1" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 40325 } } peers { permanent_uuid: "d2af6fdc8a424f808895e2bee1f3a818" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 38353 } } peers { permanent_uuid: "c7fdc2aea9e147c1a3ed5d00a2df0c86" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 34345 } }
I20260806 22:48:42.057467 22441 raft_consensus.cc:2454] T 22ea4d56c866498b87737a4c2c8a9563 P d2af6fdc8a424f808895e2bee1f3a818 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68be8534f09b4cd6b7ac9fe06b73a7e1 in term 1.
I20260806 22:48:42.058298 22029 catalog_manager.cc:5495] T 22ea4d56c866498b87737a4c2c8a9563 P 68be8534f09b4cd6b7ac9fe06b73a7e1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 68be8534f09b4cd6b7ac9fe06b73a7e1 (127.6.40.67). New cstate: current_term: 1 leader_uuid: "68be8534f09b4cd6b7ac9fe06b73a7e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68be8534f09b4cd6b7ac9fe06b73a7e1" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 40325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d2af6fdc8a424f808895e2bee1f3a818" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 38353 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7fdc2aea9e147c1a3ed5d00a2df0c86" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 34345 } health_report { overall_health: UNKNOWN } } }
I20260806 22:48:42.130829 22732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:42.127640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54752 (local address 127.6.40.67:40325)
0806 22:48:42.127730 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0806 22:48:42.127732 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:48:42.127740 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:48:42.127795 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:48:42.127796 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:48:42.127836 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:48:42.127909 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:48:42.128074 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:48:42.128588 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:48:42.129923 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:48:42.130184 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:48:42.130264 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:48:42.130447 (+ 183us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:48:42.130565 (+ 118us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:48:42.130655 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 22:48:42.130696 (+ 41us) server_negotiation.cc:299] Negotiation successful
0806 22:48:42.130718 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 22:48:42.133445 22441 raft_consensus.cc:1261] T 22ea4d56c866498b87737a4c2c8a9563 P d2af6fdc8a424f808895e2bee1f3a818 [term 1 FOLLOWER]: Refusing update from remote peer 68be8534f09b4cd6b7ac9fe06b73a7e1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:48:42.133471 22252 raft_consensus.cc:1261] T 22ea4d56c866498b87737a4c2c8a9563 P c7fdc2aea9e147c1a3ed5d00a2df0c86 [term 1 FOLLOWER]: Refusing update from remote peer 68be8534f09b4cd6b7ac9fe06b73a7e1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:48:42.133940 22745 consensus_queue.cc:1035] T 22ea4d56c866498b87737a4c2c8a9563 P 68be8534f09b4cd6b7ac9fe06b73a7e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2af6fdc8a424f808895e2bee1f3a818" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 38353 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:48:42.135895 22750 consensus_queue.cc:1035] T 22ea4d56c866498b87737a4c2c8a9563 P 68be8534f09b4cd6b7ac9fe06b73a7e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7fdc2aea9e147c1a3ed5d00a2df0c86" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 34345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260806 22:48:42.138581 22756 mvcc.cc:205] Tried to move back new op lower bound from 7315687514654965760 to 7315687514347483136. Current Snapshot: MvccSnapshot[applied={T|T < 7315687514654965760}]
I20260806 22:48:42.138958 22755 mvcc.cc:205] Tried to move back new op lower bound from 7315687514654965760 to 7315687514347483136. Current Snapshot: MvccSnapshot[applied={T|T < 7315687514654965760}]
I20260806 22:48:42.142246 22758 mvcc.cc:205] Tried to move back new op lower bound from 7315687514654965760 to 7315687514347483136. Current Snapshot: MvccSnapshot[applied={T|T < 7315687514654965760}]
I20260806 22:48:42.143244 22029 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55150:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:48:42.143353 22029 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55150:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:48:42.144196 22029 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f8293e32825848d8912a3a1b422517b8: Sending DeleteTablet for 3 replicas of tablet 22ea4d56c866498b87737a4c2c8a9563
I20260806 22:48:42.144665 22426 tablet_service.cc:1514] Processing DeleteTablet for tablet 22ea4d56c866498b87737a4c2c8a9563 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:48:42 UTC) from {username='kudu', principal='kudu/127.6.40.126@KRBTEST.COM'} at 127.0.0.1:51800
I20260806 22:48:42.145164 22613 tablet_service.cc:1514] Processing DeleteTablet for tablet 22ea4d56c866498b87737a4c2c8a9563 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:48:42 UTC) from {username='kudu', principal='kudu/127.6.40.126@KRBTEST.COM'} at 127.0.0.1:54746
I20260806 22:48:42.145673 22211 tablet_service.cc:1514] Processing DeleteTablet for tablet 22ea4d56c866498b87737a4c2c8a9563 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:48:42 UTC) from {username='kudu', principal='kudu/127.6.40.126@KRBTEST.COM'} at 127.0.0.1:39114
I20260806 22:48:42.146435 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 22133
I20260806 22:48:42.155117 22766 tablet_replica.cc:330] T 22ea4d56c866498b87737a4c2c8a9563 P 68be8534f09b4cd6b7ac9fe06b73a7e1: stopping tablet replica
I20260806 22:48:42.155294 22766 raft_consensus.cc:2229] T 22ea4d56c866498b87737a4c2c8a9563 P 68be8534f09b4cd6b7ac9fe06b73a7e1 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:48:42.155484 22766 raft_consensus.cc:2258] T 22ea4d56c866498b87737a4c2c8a9563 P 68be8534f09b4cd6b7ac9fe06b73a7e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:48:42.159299 22768 tablet_replica.cc:330] T 22ea4d56c866498b87737a4c2c8a9563 P d2af6fdc8a424f808895e2bee1f3a818: stopping tablet replica
I20260806 22:48:42.159509 22768 raft_consensus.cc:2229] T 22ea4d56c866498b87737a4c2c8a9563 P d2af6fdc8a424f808895e2bee1f3a818 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:48:42.159704 22768 raft_consensus.cc:2258] T 22ea4d56c866498b87737a4c2c8a9563 P d2af6fdc8a424f808895e2bee1f3a818 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:48:42.160411 22768 ts_tablet_manager.cc:1905] T 22ea4d56c866498b87737a4c2c8a9563 P d2af6fdc8a424f808895e2bee1f3a818: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:48:42.162688 22768 ts_tablet_manager.cc:1918] T 22ea4d56c866498b87737a4c2c8a9563 P d2af6fdc8a424f808895e2bee1f3a818: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:48:42.162833 22768 log.cc:1196] T 22ea4d56c866498b87737a4c2c8a9563 P d2af6fdc8a424f808895e2bee1f3a818: Deleting WAL directory at /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/minicluster-data/ts-1/wal/wals/22ea4d56c866498b87737a4c2c8a9563
I20260806 22:48:42.165194 22768 ts_tablet_manager.cc:1939] T 22ea4d56c866498b87737a4c2c8a9563 P d2af6fdc8a424f808895e2bee1f3a818: Deleting consensus metadata
I20260806 22:48:42.165261 22766 ts_tablet_manager.cc:1905] T 22ea4d56c866498b87737a4c2c8a9563 P 68be8534f09b4cd6b7ac9fe06b73a7e1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:48:42.167229 22766 ts_tablet_manager.cc:1918] T 22ea4d56c866498b87737a4c2c8a9563 P 68be8534f09b4cd6b7ac9fe06b73a7e1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:48:42.167313 22766 log.cc:1196] T 22ea4d56c866498b87737a4c2c8a9563 P 68be8534f09b4cd6b7ac9fe06b73a7e1: Deleting WAL directory at /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056405724609-6305-0/minicluster-data/ts-2/wal/wals/22ea4d56c866498b87737a4c2c8a9563
I20260806 22:48:42.167398 22011 catalog_manager.cc:4841] TS d2af6fdc8a424f808895e2bee1f3a818 (127.6.40.66:38353): tablet 22ea4d56c866498b87737a4c2c8a9563 (table test-table [id=445ef27eea4a4afeb519cda5e5901771]) successfully deleted
I20260806 22:48:42.167624 22766 ts_tablet_manager.cc:1939] T 22ea4d56c866498b87737a4c2c8a9563 P 68be8534f09b4cd6b7ac9fe06b73a7e1: Deleting consensus metadata
W20260806 22:48:42.167873 22009 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.40.65:34345 (error 108)
I20260806 22:48:42.168220 22010 catalog_manager.cc:4841] TS 68be8534f09b4cd6b7ac9fe06b73a7e1 (127.6.40.67:40325): tablet 22ea4d56c866498b87737a4c2c8a9563 (table test-table [id=445ef27eea4a4afeb519cda5e5901771]) successfully deleted
I20260806 22:48:42.168762 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 22327
I20260806 22:48:42.181411 22729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:42.181225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.65:34345 (local address 127.0.0.1:39128)
0806 22:48:42.181310 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0806 22:48:42.181362 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.65:34345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260806 22:48:42.181736 22009 catalog_manager.cc:4584] TS c7fdc2aea9e147c1a3ed5d00a2df0c86 (127.6.40.65:34345): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 22ea4d56c866498b87737a4c2c8a9563: Network error: Client connection negotiation failed: client connection to 127.6.40.65:34345: connect: Connection refused (error 111)
I20260806 22:48:42.191908 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 22529
I20260806 22:48:42.204111 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 21988
I20260806 22:48:42.217571 6305 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260806 22:48:42.561893 6305 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260806 22:48:42.562007 6305 mini_ranger.cc:66] Stopping Ranger...
I20260806 22:48:42.916565 6305 mini_ranger.cc:68] Stopped Ranger
2026-08-06 22:48:42.916 UTC [21382] LOG: received smart shutdown request
2026-08-06 22:48:42.921 UTC [21382] LOG: background worker "logical replication launcher" (PID 21395) exited with exit code 1
2026-08-06 22:48:42.921 UTC [21388] LOG: shutting down
2026-08-06 22:48:43.094 UTC [21382] LOG: database system is shut down
2026-08-06T22:48:43Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (54657 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:48:43 dist-test-slave-zh2d krb5kdc[22782](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:48:43 dist-test-slave-zh2d krb5kdc[22782](info): set up 2 sockets
Aug 06 22:48:43 dist-test-slave-zh2d krb5kdc[22782](info): commencing operation
krb5kdc: starting...
W20260806 22:48:44.442405 6305 mini_kdc.cc:121] Time spent starting KDC: real 1.209s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/krb5kdc/test-user.keytab.
Aug 06 22:48:44 dist-test-slave-zh2d krb5kdc[22782](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056524, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:48:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:48:44Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.6.40.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.6.40.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.6.40.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/krb5kdc/rangeradmin_127.6.40.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.6.40.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/krb5kdc/rangeradmin_127.6.40.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.6.40.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.6.40.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.6.40.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/krb5kdc/rangerlookup_127.6.40.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.6.40.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/krb5kdc/rangerlookup_127.6.40.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.6.40.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.6.40.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.6.40.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/krb5kdc/HTTP_127.6.40.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.6.40.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/krb5kdc/HTTP_127.6.40.84@KRBTEST.COM.keytab.
I20260806 22:48:44.619289 6305 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task_hhfSb/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/postgres -l logfile start
2026-08-06 22:48:47.201 UTC [22818] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-06 22:48:47.201 UTC [22818] LOG: listening on IPv4 address "127.6.40.84", port 37745
2026-08-06 22:48:47.205 UTC [22818] LOG: listening on Unix socket "/tmp/.s.PGSQL.37745"
2026-08-06 22:48:47.228 UTC [22823] LOG: database system was shut down at 2026-08-06 22:48:45 UTC
2026-08-06 22:48:47.232 UTC [22818] LOG: database system is ready to accept connections
I20260806 22:48:48.368276 6305 mini_postgres.cc:96] Postgres bound to 37745
2026-08-06 22:48:48.375 UTC [22835] FATAL: database "slave" does not exist
127.6.40.84:37745 - accepting connections
I20260806 22:48:48.375828 6305 mini_ranger.cc:161] Starting Ranger...
I20260806 22:48:48.391206 6305 mini_ranger.cc:84] Created miniranger Postgres user
I20260806 22:48:48.898025 6305 mini_ranger.cc:87] Created ranger Postgres database
I20260806 22:48:48.898128 6305 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ranger-admin
2026-08-06 22:48:49,189 [I] DB FLAVOR :POSTGRES
2026-08-06 22:48:49,190 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 22:48:49,190 [I] Checking connection..
2026-08-06 22:48:49,190 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_hhfSb/build/release/bin/postgresql.jar:/tmp/dist-test-task_hhfSb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.40.84:37745/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-06 22:48:49,466 [I] Checking connection passed.
2026-08-06 22:48:49,467 [I] --------- Verifying version history table ---------
2026-08-06 22:48:49,467 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_hhfSb/build/release/bin/postgresql.jar:/tmp/dist-test-task_hhfSb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.40.84:37745/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 22:48:49,741 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 22:48:49,742 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_hhfSb/build/release/bin/postgresql.jar:/tmp/dist-test-task_hhfSb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.40.84:37745/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 22:48:50,029 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 22:48:50,029 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-06 22:48:50,029 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_hhfSb/build/release/bin/postgresql.jar:/tmp/dist-test-task_hhfSb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.40.84:37745/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task_hhfSb/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-06 22:48:50.292 UTC [22916] WARNING: there is no transaction in progress
2026-08-06 22:48:50,300 [I] create_dbversion_catalog.sql file imported successfully
2026-08-06 22:48:50,300 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_hhfSb/build/release/bin/postgresql.jar:/tmp/dist-test-task_hhfSb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.40.84:37745/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 22:48:50,575 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-06 22:48:50,576 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 22:48:50,576 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_hhfSb/build/release/bin/postgresql.jar:/tmp/dist-test-task_hhfSb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.40.84:37745/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-06 22:48:50,863 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_hhfSb/build/release/bin/postgresql.jar:/tmp/dist-test-task_hhfSb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.40.84:37745/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-06 22:48:51,105 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_hhfSb/build/release/bin/postgresql.jar:/tmp/dist-test-task_hhfSb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.40.84:37745/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-zh2d.c.gcp-upstream.internal','N') ;"
2026-08-06 22:48:51,356 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_hhfSb/build/release/bin/postgresql.jar:/tmp/dist-test-task_hhfSb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.40.84:37745/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 22:48:51,630 [I] Table x_portal_user does not exist in database ranger
2026-08-06 22:48:51,631 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_hhfSb/build/release/bin/postgresql.jar:/tmp/dist-test-task_hhfSb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.40.84:37745/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 22:48:51,898 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-06 22:48:51,898 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_hhfSb/build/release/bin/postgresql.jar:/tmp/dist-test-task_hhfSb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.40.84:37745/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 22:48:52,174 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_hhfSb/build/release/bin/postgresql.jar:/tmp/dist-test-task_hhfSb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.40.84:37745/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 22:48:52,465 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-06 22:48:52,465 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_hhfSb/build/release/bin/postgresql.jar:/tmp/dist-test-task_hhfSb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.40.84:37745/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task_hhfSb/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-06 22:48:53.451 UTC [23068] WARNING: there is no transaction in progress
2026-08-06 22:48:53.469 UTC [23068] WARNING: there is no transaction in progress
2026-08-06 22:48:53.486 UTC [23068] WARNING: there is no transaction in progress
2026-08-06 22:48:53.503 UTC [23068] WARNING: there is no transaction in progress
2026-08-06 22:48:53.523 UTC [23068] WARNING: there is no transaction in progress
2026-08-06 22:48:53.545 UTC [23068] WARNING: there is no transaction in progress
2026-08-06 22:48:53.608 UTC [23068] WARNING: there is no transaction in progress
2026-08-06 22:48:53.615 UTC [23068] WARNING: there is no transaction in progress
2026-08-06 22:48:53.630 UTC [23068] WARNING: there is no transaction in progress
2026-08-06 22:48:53.642 UTC [23068] WARNING: there is no transaction in progress
2026-08-06 22:48:53.653 UTC [23068] WARNING: there is no transaction in progress
2026-08-06 22:48:53.668 UTC [23068] WARNING: there is no transaction in progress
2026-08-06 22:48:53.680 UTC [23068] WARNING: there is no transaction in progress
2026-08-06 22:48:53.686 UTC [23068] WARNING: there is no transaction in progress
2026-08-06 22:48:53.693 UTC [23068] WARNING: there is no transaction in progress
2026-08-06 22:48:53.700 UTC [23068] WARNING: there is no transaction in progress
2026-08-06 22:48:54,356 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-06 22:48:54,356 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_hhfSb/build/release/bin/postgresql.jar:/tmp/dist-test-task_hhfSb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.40.84:37745/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-06 22:48:54,616 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-06 22:48:54,617 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_hhfSb/build/release/bin/postgresql.jar:/tmp/dist-test-task_hhfSb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.40.84:37745/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 22:48:54,897 [I] Table x_portal_user already exists in database 'ranger'
2026-08-06 22:48:54,898 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_hhfSb/build/release/bin/postgresql.jar:/tmp/dist-test-task_hhfSb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.40.84:37745/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 22:48:55,168 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-06 22:48:55,169 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_hhfSb/build/release/bin/postgresql.jar:/tmp/dist-test-task_hhfSb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.40.84:37745/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 22:48:55,433 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_hhfSb/build/release/bin/postgresql.jar:/tmp/dist-test-task_hhfSb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.40.84:37745/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 22:48:55,702 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_hhfSb/build/release/bin/postgresql.jar:/tmp/dist-test-task_hhfSb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.40.84:37745/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-zh2d.c.gcp-upstream.internal';"
2026-08-06 22:48:55,977 [I] CORE_DB_SCHEMA import status has been updated
2026-08-06 22:48:55,977 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_hhfSb/build/release/bin/postgresql.jar:/tmp/dist-test-task_hhfSb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.40.84:37745/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-06 22:48:56,237 [I] DB_PATCHES have already been applied
I20260806 22:48:56.243441 6305 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ranger-admin:/tmp/dist-test-task_hhfSb/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task_hhfSb/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task_hhfSb/build/release/bin/postgresql.jar:/tmp/dist-test-task_hhfSb/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260806 22:48:56.243510 6305 mini_ranger.cc:193] Using host: 127.6.40.84
I20260806 22:48:56.247464 6305 mini_ranger.cc:227] Ranger admin URL: http://127.6.40.84:58363
Aug 06, 2026 10:48:56 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 10:48:56 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 10:48:56 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ranger-admin/ews/webapp
Aug 06, 2026 10:48:56 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 06, 2026 10:48:56 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 06, 2026 10:48:57 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ranger-admin/ews/webapp].
Aug 06, 2026 10:48:57 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260806 22:48:57.624486 6305 mini_ranger.cc:160] Time spent starting Ranger: real 9.249s user 0.000s sys 0.011s
Aug 06, 2026 10:49:19 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 06, 2026 10:49:19 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 06, 2026 10:49:19 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 06, 2026 10:49:19 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 06, 2026 10:49:19 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 06, 2026 10:49:19 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 06, 2026 10:49:19 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 06, 2026 10:49:19 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 06, 2026 10:49:19 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 06, 2026 10:49:19 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 06, 2026 10:49:19 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 06, 2026 10:49:19 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 06, 2026 10:49:19 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 06, 2026 10:49:19 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 06, 2026 10:49:19 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 06, 2026 10:49:19 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 06, 2026 10:49:19 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 06, 2026 10:49:20 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260806 22:49:20.834546 6305 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.6.40.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.6.40.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.6.40.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/krb5kdc/rangerkms_127.6.40.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.6.40.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/krb5kdc/rangerkms_127.6.40.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.6.40.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.6.40.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.6.40.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/krb5kdc/HTTP_127.6.40.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.6.40.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/krb5kdc/HTTP_127.6.40.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 06 22:49:20 dist-test-slave-zh2d krb5kdc[22782](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056560, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260806 22:49:20.916592 6305 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260806 22:49:20.927296 6305 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260806 22:49:21.823297 6305 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260806 22:49:21.823391 6305 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ranger-kms
I20260806 22:49:21.823412 6305 mini_ranger_kms.cc:227] Using postgres at 127.6.40.84:37745
2026-08-06 22:49:22,015 [I] DB FLAVOR :POSTGRES
2026-08-06 22:49:22,015 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 22:49:22,015 [I] Checking connection
2026-08-06 22:49:22,015 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_hhfSb/build/release/bin/postgresql.jar:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.40.84:37745/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-06 22:49:22,254 [I] connection success
2026-08-06 22:49:22,254 [I] --------- Verifying Ranger DB tables ---------
2026-08-06 22:49:22,254 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-06 22:49:22,255 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_hhfSb/build/release/bin/postgresql.jar:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.40.84:37745/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-06 22:49:22,515 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-06 22:49:22,515 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 22:49:22,515 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-06 22:49:22,515 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_hhfSb/build/release/bin/postgresql.jar:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.40.84:37745/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-06 22:49:22,797 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260806 22:49:23.353730 6305 mini_ranger_kms.cc:310] Created kms service
I20260806 22:49:23.575258 6305 mini_ranger_kms.cc:326] Created kudu user
I20260806 22:49:23.618685 6305 mini_ranger_kms.cc:343] Created rangerkms user
Aug 06 22:49:23 dist-test-slave-zh2d krb5kdc[22782](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056560, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.6.40.84@KRBTEST.COM
I20260806 22:49:24.027937 6305 mini_ranger_kms.cc:384] Added ranger policy
I20260806 22:49:24.028061 6305 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ranger-kms:/tmp/dist-test-task_hhfSb/build/release/bin/postgresql.jar:/tmp/dist-test-task_hhfSb/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task_hhfSb/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260806 22:49:24.028081 6305 mini_ranger_kms.cc:240] Using host: 127.6.40.84
I20260806 22:49:24.032212 6305 mini_ranger_kms.cc:276] Ranger KMS PID: 23322
I20260806 22:49:24.032305 6305 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.6.40.84:47851
Aug 06, 2026 10:49:25 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 10:49:25 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 10:49:25 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 06, 2026 10:49:25 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 06, 2026 10:49:25 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ranger-kms/ews/webapp].
Aug 06, 2026 10:49:25 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 06 22:49:26 dist-test-slave-zh2d krb5kdc[22782](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786056566, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.6.40.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:49:26.742151 6305 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.826s user 0.002s sys 0.172s
I20260806 22:49:26.742267 6305 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260806 22:49:26.742323 6305 mini_ranger_kms.cc:401] 127.6.40.84:47851/kms/v1/keys
Aug 06 22:49:26 dist-test-slave-zh2d krb5kdc[22782](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056566, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.6.40.84@KRBTEST.COM for HTTP/127.6.40.84@KRBTEST.COM
Aug 06 22:49:27 dist-test-slave-zh2d krb5kdc[22782](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056566, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.6.40.84@KRBTEST.COM for HTTP/127.6.40.84@KRBTEST.COM
Aug 06 22:49:29 dist-test-slave-zh2d krb5kdc[22782](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056569, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.6.40.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.40.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.40.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab.
Entry for principal kudu/127.6.40.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab.
I20260806 22:49:29.879933 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.126:34011
--webserver_interface=127.6.40.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab
--principal=kudu/127.6.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:39351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.40.126:34011
--ranger_config_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.40.84:47851/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:49:29.982606 23404 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:49:29.982816 23404 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:49:29.982838 23404 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:49:29.983973 23404 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:49:29.984014 23404 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:49:29.984028 23404 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:49:29.984041 23404 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:49:29.984053 23404 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:49:29.985682 23404 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:39351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/master-0/wal
--ranger_kms_url=127.6.40.84:47851/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.6.40.126:34011
--ranger_config_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab
--principal=kudu/127.6.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.126:34011
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23404
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:49:29.985921 23404 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:49:29.986188 23404 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:49:29.991467 23414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:29.991546 23417 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:29.991418 23412 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:29.991524 23404 server_base.cc:1034] running on GCE node
I20260806 22:49:29.991870 23404 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:49:29.992239 23404 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:49:29.993392 23404 hybrid_clock.cc:648] HybridClock initialized: now 1786056569993381 us; error 29 us; skew 500 ppm
Aug 06 22:49:29 dist-test-slave-zh2d krb5kdc[22782](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056569, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:49:29.996349 23404 init.cc:377] Logged in from keytab as kudu/127.6.40.126@KRBTEST.COM (short username kudu)
I20260806 22:49:29.996841 23404 webserver.cc:460] Webserver started at http://127.6.40.126:39047/ using document root <none> and password file <none>
I20260806 22:49:29.997108 23404 fs_manager.cc:359] Metadata directory not provided
I20260806 22:49:29.997149 23404 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:49:29.997287 23404 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 22:49:29 dist-test-slave-zh2d krb5kdc[22782](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056569, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.126@KRBTEST.COM for HTTP/127.6.40.84@KRBTEST.COM
I20260806 22:49:30.045441 23404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/master-0/data/instance:
uuid: "83eb24ce61b740af80765c65d53e86fd"
format_stamp: "Formatted at 2026-08-06 22:49:30 on dist-test-slave-zh2d"
server_key: "fcce0724dcf61857071a6bd192ca42ec"
server_key_iv: "3f79bf23c9ba7e20a14e5b865c5d8163"
server_key_version: "kuduclusterkey@0"
I20260806 22:49:30.045925 23404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/master-0/wal/instance:
uuid: "83eb24ce61b740af80765c65d53e86fd"
format_stamp: "Formatted at 2026-08-06 22:49:30 on dist-test-slave-zh2d"
server_key: "fcce0724dcf61857071a6bd192ca42ec"
server_key_iv: "3f79bf23c9ba7e20a14e5b865c5d8163"
server_key_version: "kuduclusterkey@0"
I20260806 22:49:30.047923 23404 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260806 22:49:30.060487 23430 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:30.060755 23404 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:49:30.060890 23404 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/master-0/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/master-0/wal
uuid: "83eb24ce61b740af80765c65d53e86fd"
format_stamp: "Formatted at 2026-08-06 22:49:30 on dist-test-slave-zh2d"
server_key: "fcce0724dcf61857071a6bd192ca42ec"
server_key_iv: "3f79bf23c9ba7e20a14e5b865c5d8163"
server_key_version: "kuduclusterkey@0"
I20260806 22:49:30.061017 23404 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:49:30.075444 23404 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:49:30.077662 23404 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:49:30.077850 23404 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:49:30.083038 23404 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.126:34011
I20260806 22:49:30.083122 23511 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.126:34011 every 8 connection(s)
I20260806 22:49:30.083488 23404 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
I20260806 22:49:30.084275 23514 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:49:30.086795 23514 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83eb24ce61b740af80765c65d53e86fd: Bootstrap starting.
I20260806 22:49:30.087409 23514 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 83eb24ce61b740af80765c65d53e86fd: Neither blocks nor log segments found. Creating new log.
I20260806 22:49:30.087744 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 23404
I20260806 22:49:30.087836 23514 log.cc:824] T 00000000000000000000000000000000 P 83eb24ce61b740af80765c65d53e86fd: Log is configured to *not* fsync() on all Append() calls
I20260806 22:49:30.087868 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/master-0/wal/instance
I20260806 22:49:30.088093 6305 external_mini_cluster.cc:1428] Setting key d6e42d0ef6dc327d2d3041fbb8e068c6
I20260806 22:49:30.088765 23514 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83eb24ce61b740af80765c65d53e86fd: No bootstrap required, opened a new log
I20260806 22:49:30.091097 23514 raft_consensus.cc:348] T 00000000000000000000000000000000 P 83eb24ce61b740af80765c65d53e86fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83eb24ce61b740af80765c65d53e86fd" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 34011 } }
I20260806 22:49:30.091261 23514 raft_consensus.cc:374] T 00000000000000000000000000000000 P 83eb24ce61b740af80765c65d53e86fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:49:30.091329 23514 raft_consensus.cc:729] T 00000000000000000000000000000000 P 83eb24ce61b740af80765c65d53e86fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83eb24ce61b740af80765c65d53e86fd, State: Initialized, Role: FOLLOWER
I20260806 22:49:30.091495 23514 consensus_queue.cc:260] T 00000000000000000000000000000000 P 83eb24ce61b740af80765c65d53e86fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83eb24ce61b740af80765c65d53e86fd" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 34011 } }
I20260806 22:49:30.091599 23514 raft_consensus.cc:388] T 00000000000000000000000000000000 P 83eb24ce61b740af80765c65d53e86fd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:49:30.091656 23514 raft_consensus.cc:482] T 00000000000000000000000000000000 P 83eb24ce61b740af80765c65d53e86fd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:49:30.091714 23514 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 83eb24ce61b740af80765c65d53e86fd [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:49:30.092284 23514 raft_consensus.cc:504] T 00000000000000000000000000000000 P 83eb24ce61b740af80765c65d53e86fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83eb24ce61b740af80765c65d53e86fd" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 34011 } }
I20260806 22:49:30.092408 23514 leader_election.cc:302] T 00000000000000000000000000000000 P 83eb24ce61b740af80765c65d53e86fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 83eb24ce61b740af80765c65d53e86fd; no voters:
I20260806 22:49:30.092618 23514 leader_election.cc:288] T 00000000000000000000000000000000 P 83eb24ce61b740af80765c65d53e86fd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:49:30.092646 23521 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 83eb24ce61b740af80765c65d53e86fd [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:49:30.092900 23521 raft_consensus.cc:686] T 00000000000000000000000000000000 P 83eb24ce61b740af80765c65d53e86fd [term 1 LEADER]: Becoming Leader. State: Replica: 83eb24ce61b740af80765c65d53e86fd, State: Running, Role: LEADER
I20260806 22:49:30.093008 23514 sys_catalog.cc:564] T 00000000000000000000000000000000 P 83eb24ce61b740af80765c65d53e86fd [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:49:30.093075 23521 consensus_queue.cc:237] T 00000000000000000000000000000000 P 83eb24ce61b740af80765c65d53e86fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83eb24ce61b740af80765c65d53e86fd" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 34011 } }
I20260806 22:49:30.093474 23521 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83eb24ce61b740af80765c65d53e86fd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "83eb24ce61b740af80765c65d53e86fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83eb24ce61b740af80765c65d53e86fd" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 34011 } } }
I20260806 22:49:30.093640 23521 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83eb24ce61b740af80765c65d53e86fd [sys.catalog]: This master's current role is: LEADER
I20260806 22:49:30.093851 23521 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83eb24ce61b740af80765c65d53e86fd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 83eb24ce61b740af80765c65d53e86fd. Latest consensus state: current_term: 1 leader_uuid: "83eb24ce61b740af80765c65d53e86fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83eb24ce61b740af80765c65d53e86fd" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 34011 } } }
I20260806 22:49:30.093959 23521 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83eb24ce61b740af80765c65d53e86fd [sys.catalog]: This master's current role is: LEADER
I20260806 22:49:30.094138 23530 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:49:30.094727 23530 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:49:30.095227 23514 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
Aug 06 22:49:30 dist-test-slave-zh2d krb5kdc[22782](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056569, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
I20260806 22:49:30.108417 23530 catalog_manager.cc:1347] Generated new cluster ID: fcd7622f7ba84ffd810f8073f23162e4
I20260806 22:49:30.108501 23530 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:49:30.111145 23520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:30.088640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52810 (local address 127.6.40.126:34011)
0806 22:49:30.088828 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:30.088877 (+ 49us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:30.088912 (+ 35us) server_negotiation.cc:407] Connection header received
0806 22:49:30.089474 (+ 562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:30.089485 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:30.089551 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:30.089704 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:30.090542 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:30.092298 (+ 1756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:30.093437 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:30.094091 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:30.094193 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:30.098564 (+ 4371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:30.098630 (+ 66us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:30.098631 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:30.098735 (+ 104us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:30.102151 (+ 3416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:30.109566 (+ 7415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:30.109570 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:30.109570 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:30.109683 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:30.110034 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:30.110035 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:30.110035 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:30.110355 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:30.110486 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:30.110759 (+ 273us) server_negotiation.cc:299] Negotiation successful
0806 22:49:30.110923 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":143,"thread_start_us":81,"threads_started":1}
I20260806 22:49:30.149430 23530 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:49:30.151147 23530 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:49:30.169163 23530 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 83eb24ce61b740af80765c65d53e86fd: Generated new TSK 0
I20260806 22:49:30.169487 23530 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.40.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.40.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.40.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.65.keytab.
Entry for principal kudu/127.6.40.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.65.keytab.
I20260806 22:49:30.919941 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.40.84:47851/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.65:0
--local_ip_for_outbound_sockets=127.6.40.65
--webserver_interface=127.6.40.65
--webserver_port=0
--tserver_master_addrs=127.6.40.126:34011
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.65.keytab
--principal=kudu/127.6.40.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:39351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:49:31.042096 23573 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:49:31.042450 23573 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:49:31.042522 23573 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:49:31.044221 23573 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:49:31.044325 23573 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:49:31.044425 23573 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.65
I20260806 22:49:31.046648 23573 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:39351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.6.40.84:47851/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.65.keytab
--principal=kudu/127.6.40.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.40.65
--webserver_port=0
--tserver_master_addrs=127.6.40.126:34011
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23573
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.65
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:49:31.047022 23573 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:49:31.047305 23573 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:49:31.053812 23581 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:31.053848 23585 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:31.054080 23580 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:31.054150 23573 server_base.cc:1034] running on GCE node
I20260806 22:49:31.054358 23573 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:49:31.054631 23573 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:49:31.055816 23573 hybrid_clock.cc:648] HybridClock initialized: now 1786056571055792 us; error 36 us; skew 500 ppm
Aug 06 22:49:31 dist-test-slave-zh2d krb5kdc[22782](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056571, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:49:31.059257 23573 init.cc:377] Logged in from keytab as kudu/127.6.40.65@KRBTEST.COM (short username kudu)
I20260806 22:49:31.059806 23573 webserver.cc:460] Webserver started at http://127.6.40.65:36445/ using document root <none> and password file <none>
I20260806 22:49:31.060091 23573 fs_manager.cc:359] Metadata directory not provided
I20260806 22:49:31.060159 23573 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:49:31.060289 23573 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 22:49:31 dist-test-slave-zh2d krb5kdc[22782](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056571, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.65@KRBTEST.COM for HTTP/127.6.40.84@KRBTEST.COM
I20260806 22:49:31.075165 23573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ts-0/data/instance:
uuid: "f441ce45775e4780809b60f28f279128"
format_stamp: "Formatted at 2026-08-06 22:49:31 on dist-test-slave-zh2d"
server_key: "2bf9d74a903e519de37df4a5d391a557"
server_key_iv: "9f8dd071b7c0924015aa6b8b8c0c7f8f"
server_key_version: "kuduclusterkey@0"
I20260806 22:49:31.075722 23573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ts-0/wal/instance:
uuid: "f441ce45775e4780809b60f28f279128"
format_stamp: "Formatted at 2026-08-06 22:49:31 on dist-test-slave-zh2d"
server_key: "2bf9d74a903e519de37df4a5d391a557"
server_key_iv: "9f8dd071b7c0924015aa6b8b8c0c7f8f"
server_key_version: "kuduclusterkey@0"
I20260806 22:49:31.078176 23573 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260806 22:49:31.098135 23597 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:31.098407 23573 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:49:31.098538 23573 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ts-0/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ts-0/wal
uuid: "f441ce45775e4780809b60f28f279128"
format_stamp: "Formatted at 2026-08-06 22:49:31 on dist-test-slave-zh2d"
server_key: "2bf9d74a903e519de37df4a5d391a557"
server_key_iv: "9f8dd071b7c0924015aa6b8b8c0c7f8f"
server_key_version: "kuduclusterkey@0"
I20260806 22:49:31.098652 23573 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:49:31.116873 23573 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:49:31.119473 23573 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:49:31.119678 23573 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:49:31.119976 23573 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:49:31.120508 23573 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:49:31.120621 23573 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:31.120731 23573 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:49:31.120811 23573 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:31.130373 23573 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.65:45817
I20260806 22:49:31.130892 23573 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
I20260806 22:49:31.132017 23758 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.65:45817 every 8 connection(s)
I20260806 22:49:31.137563 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 23573
I20260806 22:49:31.137678 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ts-0/wal/instance
I20260806 22:49:31.137894 6305 external_mini_cluster.cc:1428] Setting key 01d3fd60ba147bb7c957de8ff9bb8f7d
Aug 06 22:49:31 dist-test-slave-zh2d krb5kdc[22782](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056571, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.65@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
I20260806 22:49:31.146318 23761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:31.132193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:34011 (local address 127.6.40.65:55917)
0806 22:49:31.132396 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:31.132444 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:31.133097 (+ 653us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:31.133418 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:31.133423 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:31.133662 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:49:31.133931 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:31.133937 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:31.134965 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:31.134966 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:31.136345 (+ 1379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:31.136349 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:31.136960 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:31.136961 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:31.137122 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:31.138459 (+ 1337us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:49:31.138471 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:49:31.142224 (+ 3753us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:49:31.144506 (+ 2282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:31.144515 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:31.144517 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:31.144867 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:31.145117 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:31.145118 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:31.145118 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:31.145264 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:31.145570 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:49:31.145578 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:49:31.145764 (+ 186us) client_negotiation.cc:769] Sending connection context
0806 22:49:31.145880 (+ 116us) client_negotiation.cc:240] Negotiation successful
0806 22:49:31.146066 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":102,"threads_started":1}
I20260806 22:49:31.146939 23763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:31.132578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.65:55917 (local address 127.6.40.126:34011)
0806 22:49:31.132864 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:31.132866 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:31.132880 (+ 14us) server_negotiation.cc:407] Connection header received
0806 22:49:31.133223 (+ 343us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:31.133225 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:31.133241 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:31.133334 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:31.134041 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:31.134845 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:31.136453 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:31.136758 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:31.136830 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:31.142350 (+ 5520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:31.142352 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:31.142353 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:31.142364 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:31.144396 (+ 2032us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:31.144962 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:31.144964 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:31.144964 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:31.145045 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:31.145339 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:31.145340 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:31.145340 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:31.145481 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:31.145542 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:31.146731 (+ 1189us) server_negotiation.cc:299] Negotiation successful
0806 22:49:31.146827 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":240,"thread_start_us":136,"threads_started":1}
I20260806 22:49:31.147861 23759 heartbeater.cc:347] Connected to a master server at 127.6.40.126:34011
I20260806 22:49:31.147969 23759 heartbeater.cc:464] Registering TS with master...
I20260806 22:49:31.148197 23759 heartbeater.cc:511] Master 127.6.40.126:34011 requested a full tablet report, sending...
I20260806 22:49:31.148885 23455 ts_manager.cc:194] Registered new tserver with Master: f441ce45775e4780809b60f28f279128 (127.6.40.65:45817)
I20260806 22:49:31.149783 23455 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.40.65@KRBTEST.COM'} at 127.6.40.65:55917
WARNING: no policy specified for kudu/127.6.40.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.40.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.40.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.66.keytab.
Entry for principal kudu/127.6.40.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.66.keytab.
I20260806 22:49:31.186100 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.40.84:47851/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.66:0
--local_ip_for_outbound_sockets=127.6.40.66
--webserver_interface=127.6.40.66
--webserver_port=0
--tserver_master_addrs=127.6.40.126:34011
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.66.keytab
--principal=kudu/127.6.40.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:39351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:49:31.316088 23771 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:49:31.316502 23771 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:49:31.316607 23771 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:49:31.318517 23771 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:49:31.318651 23771 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:49:31.318751 23771 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.66
I20260806 22:49:31.321357 23771 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:39351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.6.40.84:47851/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.66.keytab
--principal=kudu/127.6.40.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.40.66
--webserver_port=0
--tserver_master_addrs=127.6.40.126:34011
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23771
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.66
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:49:31.321874 23771 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:49:31.322252 23771 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:49:31.330853 23783 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:31.331099 23771 server_base.cc:1034] running on GCE node
W20260806 22:49:31.332973 23781 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:31.336210 23780 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:31.336651 23771 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:49:31.337121 23771 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:49:31.338384 23771 hybrid_clock.cc:648] HybridClock initialized: now 1786056571338355 us; error 48 us; skew 500 ppm
Aug 06 22:49:31 dist-test-slave-zh2d krb5kdc[22782](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056571, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:49:31.342737 23771 init.cc:377] Logged in from keytab as kudu/127.6.40.66@KRBTEST.COM (short username kudu)
I20260806 22:49:31.343528 23771 webserver.cc:460] Webserver started at http://127.6.40.66:39761/ using document root <none> and password file <none>
I20260806 22:49:31.343915 23771 fs_manager.cc:359] Metadata directory not provided
I20260806 22:49:31.344058 23771 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:49:31.344345 23771 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 22:49:31 dist-test-slave-zh2d krb5kdc[22782](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056571, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.66@KRBTEST.COM for HTTP/127.6.40.84@KRBTEST.COM
I20260806 22:49:31.359561 23771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ts-1/data/instance:
uuid: "4a7262b9033a4215aadce65d578d154e"
format_stamp: "Formatted at 2026-08-06 22:49:31 on dist-test-slave-zh2d"
server_key: "886016ee97c51fe363f1b153854b0d9f"
server_key_iv: "c08960899202b32752179e4bcd0a7cc5"
server_key_version: "kuduclusterkey@0"
I20260806 22:49:31.360162 23771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ts-1/wal/instance:
uuid: "4a7262b9033a4215aadce65d578d154e"
format_stamp: "Formatted at 2026-08-06 22:49:31 on dist-test-slave-zh2d"
server_key: "886016ee97c51fe363f1b153854b0d9f"
server_key_iv: "c08960899202b32752179e4bcd0a7cc5"
server_key_version: "kuduclusterkey@0"
I20260806 22:49:31.364008 23771 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
Aug 06 22:49:31 dist-test-slave-zh2d krb5kdc[22782](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786056571, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:49:31.394236 23793 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:31.397176 23771 fs_manager.cc:725] Time spent opening block manager: real 0.012s user 0.001s sys 0.000s
I20260806 22:49:31.397294 23771 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ts-1/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ts-1/wal
uuid: "4a7262b9033a4215aadce65d578d154e"
format_stamp: "Formatted at 2026-08-06 22:49:31 on dist-test-slave-zh2d"
server_key: "886016ee97c51fe363f1b153854b0d9f"
server_key_iv: "c08960899202b32752179e4bcd0a7cc5"
server_key_version: "kuduclusterkey@0"
I20260806 22:49:31.397416 23771 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:49:31.445547 23771 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:49:31.448012 23771 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:49:31.448227 23771 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:49:31.448582 23771 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:49:31.448997 23771 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:49:31.449050 23771 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:31.449111 23771 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:49:31.449141 23771 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:31.457513 23771 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.66:45347
I20260806 22:49:31.458034 23771 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
I20260806 22:49:31.459461 23956 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.66:45347 every 8 connection(s)
I20260806 22:49:31.464663 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 23771
I20260806 22:49:31.464772 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ts-1/wal/instance
I20260806 22:49:31.465019 6305 external_mini_cluster.cc:1428] Setting key a24a3cc4bdef35c949db9b79af6127b5
WARNING: no policy specified for kudu/127.6.40.67@KRBTEST.COM; defaulting to no policy
Aug 06 22:49:31 dist-test-slave-zh2d krb5kdc[22782](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056571, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.66@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.40.67@KRBTEST.COM" created.
I20260806 22:49:31.484854 23763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:31.459706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.66:58775 (local address 127.6.40.126:34011)
0806 22:49:31.459761 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:31.459763 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:31.463595 (+ 3832us) server_negotiation.cc:407] Connection header received
0806 22:49:31.464246 (+ 651us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:31.464247 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:31.464262 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:31.464311 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:31.465682 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:31.466438 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:31.476921 (+ 10483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:31.477272 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:31.477384 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:31.481006 (+ 3622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:31.481008 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:31.481009 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:31.481017 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:31.482928 (+ 1911us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:31.483693 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:31.483695 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:31.483695 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:31.483766 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:31.484052 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:31.484054 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:31.484054 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:31.484220 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:31.484291 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:31.484629 (+ 338us) server_negotiation.cc:299] Negotiation successful
0806 22:49:31.484733 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 22:49:31.485057 23959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:31.459706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:34011 (local address 127.6.40.66:58775)
0806 22:49:31.463408 (+ 3702us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:31.463465 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:31.464103 (+ 638us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:31.464404 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:31.464407 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:31.464730 (+ 323us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:49:31.465573 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:31.465581 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:31.468417 (+ 2836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:31.468418 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:31.469755 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:31.469759 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:31.477419 (+ 7660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:31.477421 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:31.477599 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:31.478246 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:49:31.478258 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:49:31.480889 (+ 2631us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:49:31.483289 (+ 2400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:31.483296 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:31.483299 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:31.483610 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:31.483826 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:31.483827 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:31.483827 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:31.483981 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:31.484281 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:49:31.484288 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:49:31.484480 (+ 192us) client_negotiation.cc:769] Sending connection context
0806 22:49:31.484595 (+ 115us) client_negotiation.cc:240] Negotiation successful
0806 22:49:31.484760 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3647,"thread_start_us":85,"threads_started":1}
I20260806 22:49:31.485782 23957 heartbeater.cc:347] Connected to a master server at 127.6.40.126:34011
I20260806 22:49:31.485882 23957 heartbeater.cc:464] Registering TS with master...
I20260806 22:49:31.486130 23957 heartbeater.cc:511] Master 127.6.40.126:34011 requested a full tablet report, sending...
I20260806 22:49:31.486609 23455 ts_manager.cc:194] Registered new tserver with Master: 4a7262b9033a4215aadce65d578d154e (127.6.40.66:45347)
I20260806 22:49:31.487164 23455 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.40.66@KRBTEST.COM'} at 127.6.40.66:58775
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.40.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.67.keytab.
Entry for principal kudu/127.6.40.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.67.keytab.
I20260806 22:49:31.505125 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.40.84:47851/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.67:0
--local_ip_for_outbound_sockets=127.6.40.67
--webserver_interface=127.6.40.67
--webserver_port=0
--tserver_master_addrs=127.6.40.126:34011
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.67.keytab
--principal=kudu/127.6.40.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:39351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:49:31.631274 23965 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:49:31.631665 23965 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:49:31.631772 23965 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:49:31.634009 23965 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:49:31.634137 23965 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:49:31.634236 23965 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.67
I20260806 22:49:31.636650 23965 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:39351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.6.40.84:47851/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.67.keytab
--principal=kudu/127.6.40.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.40.67
--webserver_port=0
--tserver_master_addrs=127.6.40.126:34011
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23965
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.67
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:49:31.637218 23965 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:49:31.637610 23965 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:49:31.646390 23971 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:31.648999 23974 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:31.650198 23965 server_base.cc:1034] running on GCE node
W20260806 22:49:31.650818 23972 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:31.651196 23965 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:49:31.651507 23965 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:49:31.654795 23965 hybrid_clock.cc:648] HybridClock initialized: now 1786056571654758 us; error 48 us; skew 500 ppm
Aug 06 22:49:31 dist-test-slave-zh2d krb5kdc[22782](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056571, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:49:31.658443 23965 init.cc:377] Logged in from keytab as kudu/127.6.40.67@KRBTEST.COM (short username kudu)
I20260806 22:49:31.659052 23965 webserver.cc:460] Webserver started at http://127.6.40.67:46659/ using document root <none> and password file <none>
I20260806 22:49:31.659327 23965 fs_manager.cc:359] Metadata directory not provided
I20260806 22:49:31.659392 23965 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:49:31.659536 23965 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 22:49:31 dist-test-slave-zh2d krb5kdc[22782](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056571, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.67@KRBTEST.COM for HTTP/127.6.40.84@KRBTEST.COM
I20260806 22:49:31.698997 23965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ts-2/data/instance:
uuid: "92c11be029d24732a8a680f7ee614722"
format_stamp: "Formatted at 2026-08-06 22:49:31 on dist-test-slave-zh2d"
server_key: "aaad48633b563f2ffd74efa34555929a"
server_key_iv: "9b0e7fad6a1074fdd545c4def00028dc"
server_key_version: "kuduclusterkey@0"
I20260806 22:49:31.699528 23965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ts-2/wal/instance:
uuid: "92c11be029d24732a8a680f7ee614722"
format_stamp: "Formatted at 2026-08-06 22:49:31 on dist-test-slave-zh2d"
server_key: "aaad48633b563f2ffd74efa34555929a"
server_key_iv: "9b0e7fad6a1074fdd545c4def00028dc"
server_key_version: "kuduclusterkey@0"
I20260806 22:49:31.704213 23965 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 22:49:31.724097 23988 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:31.724323 23965 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.000s sys 0.001s
I20260806 22:49:31.724432 23965 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ts-2/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ts-2/wal
uuid: "92c11be029d24732a8a680f7ee614722"
format_stamp: "Formatted at 2026-08-06 22:49:31 on dist-test-slave-zh2d"
server_key: "aaad48633b563f2ffd74efa34555929a"
server_key_iv: "9b0e7fad6a1074fdd545c4def00028dc"
server_key_version: "kuduclusterkey@0"
I20260806 22:49:31.724568 23965 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:49:31.747140 23965 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:49:31.750113 23965 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:49:31.750293 23965 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:49:31.750602 23965 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:49:31.751106 23965 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:49:31.751163 23965 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:31.751207 23965 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:49:31.751235 23965 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:31.778277 23965 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.67:40151
I20260806 22:49:31.778609 24152 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.67:40151 every 8 connection(s)
I20260806 22:49:31.778800 23965 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
I20260806 22:49:31.788934 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 23965
I20260806 22:49:31.789064 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ts-2/wal/instance
I20260806 22:49:31.789443 6305 external_mini_cluster.cc:1428] Setting key 80876249117c1505d75ec5896f7fb8b0
Aug 06 22:49:31 dist-test-slave-zh2d krb5kdc[22782](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056571, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.67@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
I20260806 22:49:31.795153 23763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:31.779819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.67:58277 (local address 127.6.40.126:34011)
0806 22:49:31.779901 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:31.779904 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:31.780376 (+ 472us) server_negotiation.cc:407] Connection header received
0806 22:49:31.781655 (+ 1279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:31.781656 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:31.781670 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:31.781739 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:31.783103 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:31.783858 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:31.785754 (+ 1896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:31.786060 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:31.786134 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:31.790528 (+ 4394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:31.790529 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:31.790529 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:31.790539 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:31.792708 (+ 2169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:31.793489 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:31.793490 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:31.793491 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:31.793578 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:31.793974 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:31.793974 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:31.793975 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:31.794220 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:31.794300 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:31.794906 (+ 606us) server_negotiation.cc:299] Negotiation successful
0806 22:49:31.795033 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 22:49:31.795728 24155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:31.779787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:34011 (local address 127.6.40.67:58277)
0806 22:49:31.780155 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:31.780211 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:31.781500 (+ 1289us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:31.781869 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:31.781882 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:31.782414 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:49:31.782971 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:31.782978 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:31.783995 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:31.783997 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:31.785649 (+ 1652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:31.785653 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:31.786247 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:31.786249 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:31.786486 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:31.787313 (+ 827us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:49:31.787325 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:49:31.790374 (+ 3049us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:49:31.792825 (+ 2451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:31.792834 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:31.792871 (+ 37us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:31.793349 (+ 478us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:31.793699 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:31.793700 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:31.793701 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:31.793877 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:31.794286 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:49:31.794302 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:49:31.794665 (+ 363us) client_negotiation.cc:769] Sending connection context
0806 22:49:31.794860 (+ 195us) client_negotiation.cc:240] Negotiation successful
0806 22:49:31.795057 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":205,"threads_started":1}
I20260806 22:49:31.796756 24153 heartbeater.cc:347] Connected to a master server at 127.6.40.126:34011
I20260806 22:49:31.796914 24153 heartbeater.cc:464] Registering TS with master...
I20260806 22:49:31.797154 24153 heartbeater.cc:511] Master 127.6.40.126:34011 requested a full tablet report, sending...
I20260806 22:49:31.797711 23455 ts_manager.cc:194] Registered new tserver with Master: 92c11be029d24732a8a680f7ee614722 (127.6.40.67:40151)
I20260806 22:49:31.798362 23455 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.40.67@KRBTEST.COM'} at 127.6.40.67:58277
I20260806 22:49:31.801579 6305 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:49:31.802418 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 23573
I20260806 22:49:31.834043 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 23771
I20260806 22:49:31.851523 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 23965
I20260806 22:49:31.865525 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 23404
I20260806 22:49:31.886135 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.126:34011
--webserver_interface=127.6.40.126
--webserver_port=39047
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab
--principal=kudu/127.6.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:39351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.40.126:34011
--ranger_config_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.40.84:47851/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.6.40.84:47851/kms with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:49:31.972995 24158 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:49:31.973250 24158 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:49:31.973274 24158 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:49:31.974478 24158 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:49:31.974521 24158 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:49:31.974535 24158 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:49:31.974548 24158 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:49:31.974560 24158 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:49:31.976192 24158 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:39351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.6.40.84:47851/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.6.40.126:34011
--ranger_config_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab
--principal=kudu/127.6.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.126:34011
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.40.126
--webserver_port=39047
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24158
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:49:31.976455 24158 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:49:31.976691 24158 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:49:31.982923 24167 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:31.982903 24164 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:31.982899 24170 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:31.984283 24158 server_base.cc:1034] running on GCE node
I20260806 22:49:31.984597 24158 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:49:31.984990 24158 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:49:31.986183 24158 hybrid_clock.cc:648] HybridClock initialized: now 1786056571986168 us; error 29 us; skew 500 ppm
Aug 06 22:49:31 dist-test-slave-zh2d krb5kdc[22782](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056571, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:49:31.989708 24158 init.cc:377] Logged in from keytab as kudu/127.6.40.126@KRBTEST.COM (short username kudu)
I20260806 22:49:31.990254 24158 webserver.cc:460] Webserver started at http://127.6.40.126:39047/ using document root <none> and password file <none>
I20260806 22:49:31.990540 24158 fs_manager.cc:359] Metadata directory not provided
I20260806 22:49:31.990626 24158 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 22:49:32 dist-test-slave-zh2d krb5kdc[22782](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056571, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.126@KRBTEST.COM for HTTP/127.6.40.84@KRBTEST.COM
I20260806 22:49:32.036953 24158 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:49:32.037959 24181 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:32.038175 24158 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:49:32.038288 24158 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/master-0/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/master-0/wal
uuid: "83eb24ce61b740af80765c65d53e86fd"
format_stamp: "Formatted at 2026-08-06 22:49:30 on dist-test-slave-zh2d"
server_key: "fcce0724dcf61857071a6bd192ca42ec"
server_key_iv: "3f79bf23c9ba7e20a14e5b865c5d8163"
server_key_version: "kuduclusterkey@0"
I20260806 22:49:32.038836 24158 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:49:32.060155 24158 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:49:32.062237 24158 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:49:32.062407 24158 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:49:32.067503 24158 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.126:34011
I20260806 22:49:32.067544 24267 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.126:34011 every 8 connection(s)
I20260806 22:49:32.068032 24158 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
I20260806 22:49:32.069703 24268 sys_catalog.cc:263] Verifying existing consensus state
I20260806 22:49:32.070585 24268 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83eb24ce61b740af80765c65d53e86fd: Bootstrap starting.
I20260806 22:49:32.074074 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 24158
I20260806 22:49:32.074379 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.40.84:47851/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.65:45817
--local_ip_for_outbound_sockets=127.6.40.65
--tserver_master_addrs=127.6.40.126:34011
--webserver_port=36445
--webserver_interface=127.6.40.65
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.65.keytab
--principal=kudu/127.6.40.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:39351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.6.40.84:47851/kms with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:49:32.082239 24268 log.cc:824] T 00000000000000000000000000000000 P 83eb24ce61b740af80765c65d53e86fd: Log is configured to *not* fsync() on all Append() calls
I20260806 22:49:32.084631 24268 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83eb24ce61b740af80765c65d53e86fd: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 22:49:32.085006 24268 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83eb24ce61b740af80765c65d53e86fd: Bootstrap complete.
I20260806 22:49:32.087684 24268 raft_consensus.cc:348] T 00000000000000000000000000000000 P 83eb24ce61b740af80765c65d53e86fd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83eb24ce61b740af80765c65d53e86fd" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 34011 } }
I20260806 22:49:32.087940 24268 raft_consensus.cc:729] T 00000000000000000000000000000000 P 83eb24ce61b740af80765c65d53e86fd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83eb24ce61b740af80765c65d53e86fd, State: Initialized, Role: FOLLOWER
I20260806 22:49:32.088135 24268 consensus_queue.cc:260] T 00000000000000000000000000000000 P 83eb24ce61b740af80765c65d53e86fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83eb24ce61b740af80765c65d53e86fd" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 34011 } }
I20260806 22:49:32.088205 24268 raft_consensus.cc:388] T 00000000000000000000000000000000 P 83eb24ce61b740af80765c65d53e86fd [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:49:32.088272 24268 raft_consensus.cc:482] T 00000000000000000000000000000000 P 83eb24ce61b740af80765c65d53e86fd [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:49:32.088320 24268 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 83eb24ce61b740af80765c65d53e86fd [term 1 FOLLOWER]: Advancing to term 2
I20260806 22:49:32.089511 24268 raft_consensus.cc:504] T 00000000000000000000000000000000 P 83eb24ce61b740af80765c65d53e86fd [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83eb24ce61b740af80765c65d53e86fd" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 34011 } }
I20260806 22:49:32.089601 24268 leader_election.cc:302] T 00000000000000000000000000000000 P 83eb24ce61b740af80765c65d53e86fd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 83eb24ce61b740af80765c65d53e86fd; no voters:
I20260806 22:49:32.089794 24268 leader_election.cc:288] T 00000000000000000000000000000000 P 83eb24ce61b740af80765c65d53e86fd [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 22:49:32.089838 24272 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 83eb24ce61b740af80765c65d53e86fd [term 2 FOLLOWER]: Leader election won for term 2
I20260806 22:49:32.089998 24272 raft_consensus.cc:686] T 00000000000000000000000000000000 P 83eb24ce61b740af80765c65d53e86fd [term 2 LEADER]: Becoming Leader. State: Replica: 83eb24ce61b740af80765c65d53e86fd, State: Running, Role: LEADER
I20260806 22:49:32.090087 24268 sys_catalog.cc:564] T 00000000000000000000000000000000 P 83eb24ce61b740af80765c65d53e86fd [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:49:32.090206 24272 consensus_queue.cc:237] T 00000000000000000000000000000000 P 83eb24ce61b740af80765c65d53e86fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83eb24ce61b740af80765c65d53e86fd" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 34011 } }
I20260806 22:49:32.090507 24272 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83eb24ce61b740af80765c65d53e86fd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 83eb24ce61b740af80765c65d53e86fd. Latest consensus state: current_term: 2 leader_uuid: "83eb24ce61b740af80765c65d53e86fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83eb24ce61b740af80765c65d53e86fd" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 34011 } } }
I20260806 22:49:32.090512 24274 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83eb24ce61b740af80765c65d53e86fd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "83eb24ce61b740af80765c65d53e86fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83eb24ce61b740af80765c65d53e86fd" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 34011 } } }
I20260806 22:49:32.090728 24274 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83eb24ce61b740af80765c65d53e86fd [sys.catalog]: This master's current role is: LEADER
I20260806 22:49:32.091074 24282 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:49:32.091102 24272 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83eb24ce61b740af80765c65d53e86fd [sys.catalog]: This master's current role is: LEADER
I20260806 22:49:32.091643 24282 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:49:32.091774 24282 catalog_manager.cc:1259] Loaded cluster ID: fcd7622f7ba84ffd810f8073f23162e4
I20260806 22:49:32.091809 24282 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:49:32.092010 24268 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 22:49:32.093106 24282 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:49:32.093286 24282 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 83eb24ce61b740af80765c65d53e86fd: Loaded TSK: 0
I20260806 22:49:32.093497 24282 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 22:49:32.190963 24270 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:49:32.191249 24270 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:49:32.191294 24270 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:49:32.193099 24270 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:49:32.193161 24270 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:49:32.193202 24270 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.65
I20260806 22:49:32.195458 24270 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:39351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.6.40.84:47851/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.65.keytab
--principal=kudu/127.6.40.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.65:45817
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.40.65
--webserver_port=36445
--tserver_master_addrs=127.6.40.126:34011
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24270
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.65
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:49:32.195686 24270 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:49:32.195969 24270 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:49:32.203334 24320 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:32.203353 24318 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:32.203357 24316 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:32.204309 24270 server_base.cc:1034] running on GCE node
I20260806 22:49:32.204551 24270 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:49:32.204869 24270 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:49:32.206055 24270 hybrid_clock.cc:648] HybridClock initialized: now 1786056572206034 us; error 36 us; skew 500 ppm
Aug 06 22:49:32 dist-test-slave-zh2d krb5kdc[22782](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056572, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:49:32.209451 24270 init.cc:377] Logged in from keytab as kudu/127.6.40.65@KRBTEST.COM (short username kudu)
I20260806 22:49:32.210039 24270 webserver.cc:460] Webserver started at http://127.6.40.65:36445/ using document root <none> and password file <none>
I20260806 22:49:32.210378 24270 fs_manager.cc:359] Metadata directory not provided
I20260806 22:49:32.210543 24270 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 22:49:32 dist-test-slave-zh2d krb5kdc[22782](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056572, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.65@KRBTEST.COM for HTTP/127.6.40.84@KRBTEST.COM
I20260806 22:49:32.245333 24270 fs_manager.cc:709] Time spent opening directory manager: real 0.013s user 0.000s sys 0.001s
I20260806 22:49:32.246665 24328 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:32.246901 24270 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:49:32.246985 24270 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ts-0/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ts-0/wal
uuid: "f441ce45775e4780809b60f28f279128"
format_stamp: "Formatted at 2026-08-06 22:49:31 on dist-test-slave-zh2d"
server_key: "2bf9d74a903e519de37df4a5d391a557"
server_key_iv: "9f8dd071b7c0924015aa6b8b8c0c7f8f"
server_key_version: "kuduclusterkey@0"
I20260806 22:49:32.247502 24270 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:49:32.259188 24270 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:49:32.261919 24270 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:49:32.262157 24270 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:49:32.262490 24270 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:49:32.262925 24270 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:49:32.263023 24270 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:32.263134 24270 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:49:32.263217 24270 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:32.270593 24270 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.65:45817
I20260806 22:49:32.271051 24270 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
I20260806 22:49:32.272394 24506 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.65:45817 every 8 connection(s)
I20260806 22:49:32.275744 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 24270
I20260806 22:49:32.276093 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.40.84:47851/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.66:45347
--local_ip_for_outbound_sockets=127.6.40.66
--tserver_master_addrs=127.6.40.126:34011
--webserver_port=39761
--webserver_interface=127.6.40.66
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.66.keytab
--principal=kudu/127.6.40.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:39351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.6.40.84:47851/kms with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 22:49:32 dist-test-slave-zh2d krb5kdc[22782](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056572, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.65@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
I20260806 22:49:32.286835 24509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:32.273097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:34011 (local address 127.6.40.65:40701)
0806 22:49:32.273584 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:32.273629 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:32.274237 (+ 608us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:32.274930 (+ 693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:32.274935 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:32.275163 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:49:32.275489 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:32.275495 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:32.277324 (+ 1829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:32.277326 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:32.278574 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:32.278577 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:32.279564 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:32.279566 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:32.279711 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:32.280398 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:49:32.280411 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:49:32.282258 (+ 1847us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:49:32.284704 (+ 2446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:32.284711 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:32.284714 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:32.285052 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:32.285344 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:32.285345 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:32.285345 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:32.285510 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:32.286105 (+ 595us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:49:32.286113 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:49:32.286279 (+ 166us) client_negotiation.cc:769] Sending connection context
0806 22:49:32.286379 (+ 100us) client_negotiation.cc:240] Negotiation successful
0806 22:49:32.286554 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":96,"threads_started":1}
I20260806 22:49:32.287241 24511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:32.273172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.65:40701 (local address 127.6.40.126:34011)
0806 22:49:32.274477 (+ 1305us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:32.274487 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:32.274525 (+ 38us) server_negotiation.cc:407] Connection header received
0806 22:49:32.274669 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:32.274676 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:32.274731 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:32.275626 (+ 895us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:32.275734 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:32.277201 (+ 1467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:32.278688 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:32.279422 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:32.281169 (+ 1747us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:32.282384 (+ 1215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:32.282390 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:32.282390 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:32.282406 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:32.284602 (+ 2196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:32.285152 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:32.285155 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:32.285155 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:32.285256 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:32.285636 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:32.285638 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:32.285638 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:32.285999 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:32.286772 (+ 773us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:32.286855 (+ 83us) server_negotiation.cc:299] Negotiation successful
0806 22:49:32.287000 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1245,"thread_start_us":77,"threads_started":1}
I20260806 22:49:32.287985 24507 heartbeater.cc:347] Connected to a master server at 127.6.40.126:34011
I20260806 22:49:32.288082 24507 heartbeater.cc:464] Registering TS with master...
I20260806 22:49:32.288277 24507 heartbeater.cc:511] Master 127.6.40.126:34011 requested a full tablet report, sending...
W20260806 22:49:32.289235 24507 heartbeater.cc:644] Failed to heartbeat to 127.6.40.126:34011 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 22:49:32.289316 24507 heartbeater.cc:464] Registering TS with master...
I20260806 22:49:32.289390 24507 heartbeater.cc:511] Master 127.6.40.126:34011 requested a full tablet report, sending...
I20260806 22:49:32.289917 24507 heartbeater.cc:464] Registering TS with master...
I20260806 22:49:32.290004 24507 heartbeater.cc:511] Master 127.6.40.126:34011 requested a full tablet report, sending...
W20260806 22:49:32.290474 24507 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 22:49:32.399710 24512 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:49:32.399942 24512 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:49:32.399982 24512 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:49:32.401513 24512 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:49:32.401580 24512 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:49:32.401626 24512 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.66
I20260806 22:49:32.403726 24512 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:39351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.6.40.84:47851/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.66.keytab
--principal=kudu/127.6.40.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.66:45347
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.40.66
--webserver_port=39761
--tserver_master_addrs=127.6.40.126:34011
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24512
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.66
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:49:32.404006 24512 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:49:32.404239 24512 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:49:32.409761 24530 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:32.409886 24524 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:32.409935 24525 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:32.410631 24512 server_base.cc:1034] running on GCE node
I20260806 22:49:32.410871 24512 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:49:32.411170 24512 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:49:32.412360 24512 hybrid_clock.cc:648] HybridClock initialized: now 1786056572412345 us; error 29 us; skew 500 ppm
Aug 06 22:49:32 dist-test-slave-zh2d krb5kdc[22782](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056572, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:49:32.415787 24512 init.cc:377] Logged in from keytab as kudu/127.6.40.66@KRBTEST.COM (short username kudu)
I20260806 22:49:32.416402 24512 webserver.cc:460] Webserver started at http://127.6.40.66:39761/ using document root <none> and password file <none>
I20260806 22:49:32.416711 24512 fs_manager.cc:359] Metadata directory not provided
I20260806 22:49:32.416810 24512 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 22:49:32 dist-test-slave-zh2d krb5kdc[22782](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056572, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.66@KRBTEST.COM for HTTP/127.6.40.84@KRBTEST.COM
I20260806 22:49:32.433038 24512 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:49:32.434165 24541 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:32.434381 24512 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 22:49:32.434485 24512 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ts-1/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ts-1/wal
uuid: "4a7262b9033a4215aadce65d578d154e"
format_stamp: "Formatted at 2026-08-06 22:49:31 on dist-test-slave-zh2d"
server_key: "886016ee97c51fe363f1b153854b0d9f"
server_key_iv: "c08960899202b32752179e4bcd0a7cc5"
server_key_version: "kuduclusterkey@0"
I20260806 22:49:32.434958 24512 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:49:32.451409 24512 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:49:32.453991 24512 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:49:32.454178 24512 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:49:32.454433 24512 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:49:32.454804 24512 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:49:32.454857 24512 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:32.454890 24512 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:49:32.454957 24512 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:32.463164 24512 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.66:45347
I20260806 22:49:32.463258 24699 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.66:45347 every 8 connection(s)
I20260806 22:49:32.463742 24512 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
Aug 06 22:49:32 dist-test-slave-zh2d krb5kdc[22782](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056572, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.66@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
I20260806 22:49:32.473280 24511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:32.464548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.66:36103 (local address 127.6.40.126:34011)
0806 22:49:32.464633 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:32.464636 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:32.465059 (+ 423us) server_negotiation.cc:407] Connection header received
0806 22:49:32.465609 (+ 550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:32.465610 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:32.465623 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:32.465687 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:32.466240 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:32.466761 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:32.467789 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:32.468023 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:32.468110 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:32.470326 (+ 2216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:32.470327 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:32.470327 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:32.470335 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:32.471843 (+ 1508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:32.472252 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:32.472254 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:32.472254 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:32.472316 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:32.472564 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:32.472566 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:32.472566 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:32.472695 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:32.472764 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:32.473094 (+ 330us) server_negotiation.cc:299] Negotiation successful
0806 22:49:32.473187 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 22:49:32.473489 24703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:32.464499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:34011 (local address 127.6.40.66:36103)
0806 22:49:32.464935 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:32.464975 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:32.465505 (+ 530us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:32.465760 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:32.465764 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:32.465951 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:49:32.466145 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:32.466149 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:32.466876 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:32.466877 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:32.467688 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:32.467690 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:32.468148 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:32.468149 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:32.468263 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:32.468893 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:49:32.468913 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:49:32.470225 (+ 1312us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:49:32.471944 (+ 1719us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:32.471951 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:32.471953 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:32.472177 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:32.472392 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:32.472393 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:32.472393 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:32.472501 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:32.472766 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:49:32.472774 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:49:32.472955 (+ 181us) client_negotiation.cc:769] Sending connection context
0806 22:49:32.473072 (+ 117us) client_negotiation.cc:240] Negotiation successful
0806 22:49:32.473218 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":93,"threads_started":1}
I20260806 22:49:32.473891 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 24512
I20260806 22:49:32.474150 24700 heartbeater.cc:347] Connected to a master server at 127.6.40.126:34011
I20260806 22:49:32.474206 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.40.84:47851/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.67:40151
--local_ip_for_outbound_sockets=127.6.40.67
--tserver_master_addrs=127.6.40.126:34011
--webserver_port=46659
--webserver_interface=127.6.40.67
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.67.keytab
--principal=kudu/127.6.40.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:39351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.6.40.84:47851/kms with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:49:32.474260 24700 heartbeater.cc:464] Registering TS with master...
I20260806 22:49:32.474457 24700 heartbeater.cc:511] Master 127.6.40.126:34011 requested a full tablet report, sending...
W20260806 22:49:32.475184 24700 heartbeater.cc:644] Failed to heartbeat to 127.6.40.126:34011 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 22:49:32.475257 24700 heartbeater.cc:464] Registering TS with master...
I20260806 22:49:32.475324 24700 heartbeater.cc:511] Master 127.6.40.126:34011 requested a full tablet report, sending...
I20260806 22:49:32.475687 24700 heartbeater.cc:464] Registering TS with master...
I20260806 22:49:32.475767 24700 heartbeater.cc:511] Master 127.6.40.126:34011 requested a full tablet report, sending...
W20260806 22:49:32.476089 24700 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 22:49:32.579064 24704 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:49:32.579422 24704 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:49:32.579535 24704 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:49:32.580819 24704 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:49:32.580917 24704 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:49:32.581014 24704 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.67
I20260806 22:49:32.582640 24704 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:39351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.6.40.84:47851/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.67.keytab
--principal=kudu/127.6.40.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.67:40151
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.40.67
--webserver_port=46659
--tserver_master_addrs=127.6.40.126:34011
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24704
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.67
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:49:32.582907 24704 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:49:32.583206 24704 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:49:32.588258 24712 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:32.588343 24710 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:32.588307 24715 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:32.588869 24704 server_base.cc:1034] running on GCE node
I20260806 22:49:32.589092 24704 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:49:32.589355 24704 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:49:32.590502 24704 hybrid_clock.cc:648] HybridClock initialized: now 1786056572590480 us; error 41 us; skew 500 ppm
Aug 06 22:49:32 dist-test-slave-zh2d krb5kdc[22782](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056572, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:49:32.594203 24704 init.cc:377] Logged in from keytab as kudu/127.6.40.67@KRBTEST.COM (short username kudu)
I20260806 22:49:32.594781 24704 webserver.cc:460] Webserver started at http://127.6.40.67:46659/ using document root <none> and password file <none>
I20260806 22:49:32.595148 24704 fs_manager.cc:359] Metadata directory not provided
I20260806 22:49:32.595294 24704 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 22:49:32 dist-test-slave-zh2d krb5kdc[22782](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056572, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.67@KRBTEST.COM for HTTP/127.6.40.84@KRBTEST.COM
I20260806 22:49:32.614670 24704 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:49:32.617012 24728 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:32.617208 24704 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.000s
I20260806 22:49:32.617300 24704 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ts-2/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ts-2/wal
uuid: "92c11be029d24732a8a680f7ee614722"
format_stamp: "Formatted at 2026-08-06 22:49:31 on dist-test-slave-zh2d"
server_key: "aaad48633b563f2ffd74efa34555929a"
server_key_iv: "9b0e7fad6a1074fdd545c4def00028dc"
server_key_version: "kuduclusterkey@0"
I20260806 22:49:32.617830 24704 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:49:32.636696 24704 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:49:32.638932 24704 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:49:32.639137 24704 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:49:32.639431 24704 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:49:32.639880 24704 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:49:32.639938 24704 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:32.639984 24704 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:49:32.640012 24704 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:32.647905 24704 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.67:40151
I20260806 22:49:32.648329 24704 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
I20260806 22:49:32.648748 24891 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.67:40151 every 8 connection(s)
I20260806 22:49:32.651966 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 24704
Aug 06 22:49:32 dist-test-slave-zh2d krb5kdc[22782](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056572, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.67@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
I20260806 22:49:32.659128 24511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:32.649087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.67:32799 (local address 127.6.40.126:34011)
0806 22:49:32.649143 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:32.649146 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:32.649380 (+ 234us) server_negotiation.cc:407] Connection header received
0806 22:49:32.649919 (+ 539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:32.649920 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:32.649933 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:32.649993 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:32.650557 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:32.651154 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:32.652948 (+ 1794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:32.653230 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:32.653317 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:32.655769 (+ 2452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:32.655771 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:32.655771 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:32.655779 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:32.657469 (+ 1690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:32.657995 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:32.657997 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:32.657997 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:32.658063 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:32.658358 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:32.658359 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:32.658359 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:32.658497 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:32.658542 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:32.658945 (+ 403us) server_negotiation.cc:299] Negotiation successful
0806 22:49:32.659032 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 22:49:32.659894 24894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:32.649089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:34011 (local address 127.6.40.67:32799)
0806 22:49:32.649251 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:32.649295 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:32.649807 (+ 512us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:32.650068 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:32.650077 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:32.650259 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:49:32.650459 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:32.650464 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:32.651300 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:32.651301 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:32.652768 (+ 1467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:32.652772 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:32.653369 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:32.653370 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:32.653495 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:32.654088 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:49:32.654104 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:49:32.655669 (+ 1565us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:49:32.657569 (+ 1900us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:32.657577 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:32.657579 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:32.657876 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:32.658143 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:32.658143 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:32.658144 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:32.658276 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:32.658592 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:49:32.658598 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:49:32.658757 (+ 159us) client_negotiation.cc:769] Sending connection context
0806 22:49:32.659490 (+ 733us) client_negotiation.cc:240] Negotiation successful
0806 22:49:32.659663 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":88,"threads_started":1}
I20260806 22:49:32.660591 24892 heartbeater.cc:347] Connected to a master server at 127.6.40.126:34011
I20260806 22:49:32.660686 24892 heartbeater.cc:464] Registering TS with master...
I20260806 22:49:32.660889 24892 heartbeater.cc:511] Master 127.6.40.126:34011 requested a full tablet report, sending...
W20260806 22:49:32.661590 24892 heartbeater.cc:644] Failed to heartbeat to 127.6.40.126:34011 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 22:49:32.661657 24892 heartbeater.cc:464] Registering TS with master...
I20260806 22:49:32.661716 24892 heartbeater.cc:511] Master 127.6.40.126:34011 requested a full tablet report, sending...
I20260806 22:49:32.662096 24892 heartbeater.cc:464] Registering TS with master...
I20260806 22:49:32.662173 24892 heartbeater.cc:511] Master 127.6.40.126:34011 requested a full tablet report, sending...
W20260806 22:49:32.662506 24892 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 22:49:32.666796 24895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:32.652474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52818 (local address 127.6.40.126:34011)
0806 22:49:32.652695 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:32.652697 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:32.652964 (+ 267us) server_negotiation.cc:407] Connection header received
0806 22:49:32.652985 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:32.652986 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:32.653000 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:32.653091 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:32.657761 (+ 4670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:32.660078 (+ 2317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:32.661722 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:32.662039 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:32.662110 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:32.663734 (+ 1624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:32.663735 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:32.663736 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:32.663743 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:32.665542 (+ 1799us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:32.665966 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:32.665967 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:32.665967 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:32.666022 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:32.666245 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:32.666246 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:32.666246 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:32.666373 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:32.666528 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:32.666653 (+ 125us) server_negotiation.cc:299] Negotiation successful
0806 22:49:32.666723 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":88,"threads_started":1}
Aug 06 22:49:33 dist-test-slave-zh2d krb5kdc[22782](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786056573, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:49:33.290671 24507 heartbeater.cc:464] Registering TS with master...
I20260806 22:49:33.290858 24507 heartbeater.cc:511] Master 127.6.40.126:34011 requested a full tablet report, sending...
I20260806 22:49:33.291307 24212 ts_manager.cc:194] Registered new tserver with Master: f441ce45775e4780809b60f28f279128 (127.6.40.65:45817)
I20260806 22:49:33.292474 24212 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.40.65@KRBTEST.COM'} at 127.6.40.65:40701
I20260806 22:49:33.476296 24700 heartbeater.cc:464] Registering TS with master...
I20260806 22:49:33.476481 24700 heartbeater.cc:511] Master 127.6.40.126:34011 requested a full tablet report, sending...
I20260806 22:49:33.476991 24212 ts_manager.cc:194] Registered new tserver with Master: 4a7262b9033a4215aadce65d578d154e (127.6.40.66:45347)
I20260806 22:49:33.477592 24212 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.40.66@KRBTEST.COM'} at 127.6.40.66:36103
I20260806 22:49:33.662690 24892 heartbeater.cc:464] Registering TS with master...
I20260806 22:49:33.662854 24892 heartbeater.cc:511] Master 127.6.40.126:34011 requested a full tablet report, sending...
I20260806 22:49:33.663196 24212 ts_manager.cc:194] Registered new tserver with Master: 92c11be029d24732a8a680f7ee614722 (127.6.40.67:40151)
I20260806 22:49:33.663748 24212 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.40.67@KRBTEST.COM'} at 127.6.40.67:32799
I20260806 22:49:33.670353 6305 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:49:33.689215 24921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:33.672503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52822 (local address 127.6.40.126:34011)
0806 22:49:33.674797 (+ 2294us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:33.674800 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:33.675023 (+ 223us) server_negotiation.cc:407] Connection header received
0806 22:49:33.675051 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:33.675052 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:33.675066 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:33.675127 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:33.675844 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:33.676654 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:33.677732 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:33.678056 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:33.678113 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:33.685607 (+ 7494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:33.685609 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:33.685610 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:33.685618 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:33.687470 (+ 1852us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:33.688022 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:33.688023 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:33.688024 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:33.688086 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:33.688267 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:33.688268 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:33.688268 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:33.688410 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:33.688511 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:33.689003 (+ 492us) server_negotiation.cc:299] Negotiation successful
0806 22:49:33.689095 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2265,"thread_start_us":128,"threads_started":1}
I20260806 22:49:33.691795 24212 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52822:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:49:33.694188 24212 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
Aug 06 22:49:33 dist-test-slave-zh2d krb5kdc[22782](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056573, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.126@KRBTEST.COM for HTTP/127.6.40.84@KRBTEST.COM
I20260806 22:49:33.708029 24931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:33.698155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52202 (local address 127.6.40.66:45347)
0806 22:49:33.698339 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:33.698344 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:33.698936 (+ 592us) server_negotiation.cc:407] Connection header received
0806 22:49:33.699053 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:33.699056 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:33.699074 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:33.699130 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:49:33.699463 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:33.700951 (+ 1488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:33.705674 (+ 4723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:33.706839 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:33.706889 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:33.706949 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:33.707884 (+ 935us) server_negotiation.cc:299] Negotiation successful
0806 22:49:33.707933 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":80,"threads_started":1}
I20260806 22:49:33.708353 24930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:33.697849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.66:45347 (local address 127.0.0.1:52202)
0806 22:49:33.698860 (+ 1011us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:33.698871 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:33.698967 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:33.699268 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:33.699271 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:33.699273 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:49:33.699348 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:33.699350 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:33.701049 (+ 1699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:33.701051 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:33.702369 (+ 1318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:33.702372 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:33.707017 (+ 4645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:33.707018 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:33.707112 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:33.707126 (+ 14us) client_negotiation.cc:769] Sending connection context
0806 22:49:33.708178 (+ 1052us) client_negotiation.cc:240] Negotiation successful
0806 22:49:33.708251 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":992,"thread_start_us":74,"threads_started":1}
I20260806 22:49:33.710009 24928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:33.697617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.65:45817 (local address 127.0.0.1:55984)
0806 22:49:33.702870 (+ 5253us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:33.702882 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:33.702930 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:33.703228 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:33.703229 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:33.703231 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:49:33.703274 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:33.703275 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:33.704581 (+ 1306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:33.704583 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:33.707775 (+ 3192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:33.707777 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:33.709754 (+ 1977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:33.709755 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:33.709824 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:33.709829 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:49:33.709890 (+ 61us) client_negotiation.cc:240] Negotiation successful
0806 22:49:33.709929 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4964,"mutex_wait_us":769,"thread_start_us":90,"threads_started":1}
I20260806 22:49:33.710248 24929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:33.697756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55984 (local address 127.6.40.65:45817)
0806 22:49:33.702535 (+ 4779us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:33.702540 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:33.702990 (+ 450us) server_negotiation.cc:407] Connection header received
0806 22:49:33.703034 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:33.703036 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:33.703053 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:33.703126 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:49:33.703363 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:33.704481 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:33.708644 (+ 4163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:33.709650 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:33.709727 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:33.709781 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:33.710126 (+ 345us) server_negotiation.cc:299] Negotiation successful
0806 22:49:33.710172 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4733,"thread_start_us":53,"threads_started":1}
I20260806 22:49:33.710336 24594 tablet_service.cc:1467] Processing CreateTablet for tablet aeb3c8b095c140fd8c785022413cc759 (DEFAULT_TABLE table=test-table [id=0fbbb627a99c4967a90d6604fd9999c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:49:33.710738 24594 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aeb3c8b095c140fd8c785022413cc759. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:49:33.716962 24410 tablet_service.cc:1467] Processing CreateTablet for tablet aeb3c8b095c140fd8c785022413cc759 (DEFAULT_TABLE table=test-table [id=0fbbb627a99c4967a90d6604fd9999c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:49:33.717417 24410 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aeb3c8b095c140fd8c785022413cc759. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:49:33.718741 24934 tablet_bootstrap.cc:492] T aeb3c8b095c140fd8c785022413cc759 P 4a7262b9033a4215aadce65d578d154e: Bootstrap starting.
I20260806 22:49:33.719632 24934 tablet_bootstrap.cc:654] T aeb3c8b095c140fd8c785022413cc759 P 4a7262b9033a4215aadce65d578d154e: Neither blocks nor log segments found. Creating new log.
I20260806 22:49:33.720094 24934 log.cc:824] T aeb3c8b095c140fd8c785022413cc759 P 4a7262b9033a4215aadce65d578d154e: Log is configured to *not* fsync() on all Append() calls
I20260806 22:49:33.720232 24933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:33.697617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.67:40151 (local address 127.0.0.1:35522)
0806 22:49:33.698565 (+ 948us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:33.698607 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:33.698684 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:33.705384 (+ 6700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:33.705385 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:33.705388 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:49:33.705438 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:33.705439 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:33.706658 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:33.706659 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:33.718732 (+ 12073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:33.718736 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:33.719947 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:33.719948 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:33.720027 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:33.720032 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:49:33.720084 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 22:49:33.720136 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":915,"thread_start_us":79,"threads_started":1}
I20260806 22:49:33.720721 24932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:33.698154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35522 (local address 127.6.40.67:40151)
0806 22:49:33.704956 (+ 6802us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:33.704961 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:33.704973 (+ 12us) server_negotiation.cc:407] Connection header received
0806 22:49:33.705182 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:33.705186 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:33.705205 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:33.705286 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:49:33.705522 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:33.706554 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:33.718875 (+ 12321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:33.719710 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:33.719809 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:33.719870 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:33.720574 (+ 704us) server_negotiation.cc:299] Negotiation successful
0806 22:49:33.720632 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6758,"thread_start_us":89,"threads_started":1}
I20260806 22:49:33.721474 24793 tablet_service.cc:1467] Processing CreateTablet for tablet aeb3c8b095c140fd8c785022413cc759 (DEFAULT_TABLE table=test-table [id=0fbbb627a99c4967a90d6604fd9999c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:49:33.721861 24793 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aeb3c8b095c140fd8c785022413cc759. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:49:33.722011 24936 tablet_bootstrap.cc:492] T aeb3c8b095c140fd8c785022413cc759 P f441ce45775e4780809b60f28f279128: Bootstrap starting.
I20260806 22:49:33.722741 24936 tablet_bootstrap.cc:654] T aeb3c8b095c140fd8c785022413cc759 P f441ce45775e4780809b60f28f279128: Neither blocks nor log segments found. Creating new log.
I20260806 22:49:33.723165 24936 log.cc:824] T aeb3c8b095c140fd8c785022413cc759 P f441ce45775e4780809b60f28f279128: Log is configured to *not* fsync() on all Append() calls
I20260806 22:49:33.724125 24936 tablet_bootstrap.cc:492] T aeb3c8b095c140fd8c785022413cc759 P f441ce45775e4780809b60f28f279128: No bootstrap required, opened a new log
I20260806 22:49:33.724192 24936 ts_tablet_manager.cc:1397] T aeb3c8b095c140fd8c785022413cc759 P f441ce45775e4780809b60f28f279128: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:49:33.727342 24936 raft_consensus.cc:348] T aeb3c8b095c140fd8c785022413cc759 P f441ce45775e4780809b60f28f279128 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92c11be029d24732a8a680f7ee614722" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 40151 } } peers { permanent_uuid: "f441ce45775e4780809b60f28f279128" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 45817 } } peers { permanent_uuid: "4a7262b9033a4215aadce65d578d154e" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 45347 } }
I20260806 22:49:33.727514 24936 raft_consensus.cc:374] T aeb3c8b095c140fd8c785022413cc759 P f441ce45775e4780809b60f28f279128 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:49:33.727562 24936 raft_consensus.cc:729] T aeb3c8b095c140fd8c785022413cc759 P f441ce45775e4780809b60f28f279128 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f441ce45775e4780809b60f28f279128, State: Initialized, Role: FOLLOWER
I20260806 22:49:33.727705 24936 consensus_queue.cc:260] T aeb3c8b095c140fd8c785022413cc759 P f441ce45775e4780809b60f28f279128 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92c11be029d24732a8a680f7ee614722" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 40151 } } peers { permanent_uuid: "f441ce45775e4780809b60f28f279128" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 45817 } } peers { permanent_uuid: "4a7262b9033a4215aadce65d578d154e" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 45347 } }
I20260806 22:49:33.728057 24507 heartbeater.cc:503] Master 127.6.40.126:34011 was elected leader, sending a full tablet report...
I20260806 22:49:33.728045 24936 ts_tablet_manager.cc:1428] T aeb3c8b095c140fd8c785022413cc759 P f441ce45775e4780809b60f28f279128: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260806 22:49:33.728545 24940 tablet_bootstrap.cc:492] T aeb3c8b095c140fd8c785022413cc759 P 92c11be029d24732a8a680f7ee614722: Bootstrap starting.
I20260806 22:49:33.729460 24940 tablet_bootstrap.cc:654] T aeb3c8b095c140fd8c785022413cc759 P 92c11be029d24732a8a680f7ee614722: Neither blocks nor log segments found. Creating new log.
I20260806 22:49:33.729844 24940 log.cc:824] T aeb3c8b095c140fd8c785022413cc759 P 92c11be029d24732a8a680f7ee614722: Log is configured to *not* fsync() on all Append() calls
I20260806 22:49:33.731027 24940 tablet_bootstrap.cc:492] T aeb3c8b095c140fd8c785022413cc759 P 92c11be029d24732a8a680f7ee614722: No bootstrap required, opened a new log
I20260806 22:49:33.731119 24940 ts_tablet_manager.cc:1397] T aeb3c8b095c140fd8c785022413cc759 P 92c11be029d24732a8a680f7ee614722: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260806 22:49:33.732904 24940 raft_consensus.cc:348] T aeb3c8b095c140fd8c785022413cc759 P 92c11be029d24732a8a680f7ee614722 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92c11be029d24732a8a680f7ee614722" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 40151 } } peers { permanent_uuid: "f441ce45775e4780809b60f28f279128" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 45817 } } peers { permanent_uuid: "4a7262b9033a4215aadce65d578d154e" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 45347 } }
I20260806 22:49:33.733116 24940 raft_consensus.cc:374] T aeb3c8b095c140fd8c785022413cc759 P 92c11be029d24732a8a680f7ee614722 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:49:33.733166 24940 raft_consensus.cc:729] T aeb3c8b095c140fd8c785022413cc759 P 92c11be029d24732a8a680f7ee614722 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92c11be029d24732a8a680f7ee614722, State: Initialized, Role: FOLLOWER
I20260806 22:49:33.733340 24940 consensus_queue.cc:260] T aeb3c8b095c140fd8c785022413cc759 P 92c11be029d24732a8a680f7ee614722 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92c11be029d24732a8a680f7ee614722" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 40151 } } peers { permanent_uuid: "f441ce45775e4780809b60f28f279128" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 45817 } } peers { permanent_uuid: "4a7262b9033a4215aadce65d578d154e" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 45347 } }
I20260806 22:49:33.733650 24940 ts_tablet_manager.cc:1428] T aeb3c8b095c140fd8c785022413cc759 P 92c11be029d24732a8a680f7ee614722: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260806 22:49:33.733791 24892 heartbeater.cc:503] Master 127.6.40.126:34011 was elected leader, sending a full tablet report...
I20260806 22:49:33.735754 24934 tablet_bootstrap.cc:492] T aeb3c8b095c140fd8c785022413cc759 P 4a7262b9033a4215aadce65d578d154e: No bootstrap required, opened a new log
I20260806 22:49:33.735855 24934 ts_tablet_manager.cc:1397] T aeb3c8b095c140fd8c785022413cc759 P 4a7262b9033a4215aadce65d578d154e: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.000s
I20260806 22:49:33.739130 24934 raft_consensus.cc:348] T aeb3c8b095c140fd8c785022413cc759 P 4a7262b9033a4215aadce65d578d154e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92c11be029d24732a8a680f7ee614722" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 40151 } } peers { permanent_uuid: "f441ce45775e4780809b60f28f279128" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 45817 } } peers { permanent_uuid: "4a7262b9033a4215aadce65d578d154e" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 45347 } }
I20260806 22:49:33.739329 24934 raft_consensus.cc:374] T aeb3c8b095c140fd8c785022413cc759 P 4a7262b9033a4215aadce65d578d154e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:49:33.739377 24934 raft_consensus.cc:729] T aeb3c8b095c140fd8c785022413cc759 P 4a7262b9033a4215aadce65d578d154e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a7262b9033a4215aadce65d578d154e, State: Initialized, Role: FOLLOWER
I20260806 22:49:33.739531 24934 consensus_queue.cc:260] T aeb3c8b095c140fd8c785022413cc759 P 4a7262b9033a4215aadce65d578d154e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92c11be029d24732a8a680f7ee614722" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 40151 } } peers { permanent_uuid: "f441ce45775e4780809b60f28f279128" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 45817 } } peers { permanent_uuid: "4a7262b9033a4215aadce65d578d154e" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 45347 } }
I20260806 22:49:33.739845 24934 ts_tablet_manager.cc:1428] T aeb3c8b095c140fd8c785022413cc759 P 4a7262b9033a4215aadce65d578d154e: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260806 22:49:33.739969 24700 heartbeater.cc:503] Master 127.6.40.126:34011 was elected leader, sending a full tablet report...
W20260806 22:49:33.774616 24508 tablet.cc:2369] T aeb3c8b095c140fd8c785022413cc759 P f441ce45775e4780809b60f28f279128: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:49:33.790696 24943 raft_consensus.cc:482] T aeb3c8b095c140fd8c785022413cc759 P 92c11be029d24732a8a680f7ee614722 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:49:33.790861 24943 raft_consensus.cc:504] T aeb3c8b095c140fd8c785022413cc759 P 92c11be029d24732a8a680f7ee614722 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92c11be029d24732a8a680f7ee614722" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 40151 } } peers { permanent_uuid: "f441ce45775e4780809b60f28f279128" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 45817 } } peers { permanent_uuid: "4a7262b9033a4215aadce65d578d154e" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 45347 } }
I20260806 22:49:33.791283 24943 leader_election.cc:288] T aeb3c8b095c140fd8c785022413cc759 P 92c11be029d24732a8a680f7ee614722 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f441ce45775e4780809b60f28f279128 (127.6.40.65:45817), 4a7262b9033a4215aadce65d578d154e (127.6.40.66:45347)
I20260806 22:49:33.796167 24929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:33.791755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.67:41033 (local address 127.6.40.65:45817)
0806 22:49:33.791818 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:33.791821 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:33.791958 (+ 137us) server_negotiation.cc:407] Connection header received
0806 22:49:33.792088 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:33.792090 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:33.792105 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:33.792138 (+ 33us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:49:33.792399 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:33.793676 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:33.795144 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:33.795715 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:33.795769 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:33.795781 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:33.796041 (+ 260us) server_negotiation.cc:299] Negotiation successful
0806 22:49:33.796087 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 22:49:33.796406 24948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:33.791463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.65:45817 (local address 127.6.40.67:41033)
0806 22:49:33.791858 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:33.791874 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:33.791993 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:33.792213 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:33.792213 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:33.792216 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:49:33.792310 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:33.792312 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:33.793783 (+ 1471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:33.793784 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:33.795025 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:33.795029 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:33.795863 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:33.795864 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:33.795941 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:33.795958 (+ 17us) client_negotiation.cc:769] Sending connection context
0806 22:49:33.796264 (+ 306us) client_negotiation.cc:240] Negotiation successful
0806 22:49:33.796309 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":119,"threads_started":1}
I20260806 22:49:33.796806 24441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeb3c8b095c140fd8c785022413cc759" candidate_uuid: "92c11be029d24732a8a680f7ee614722" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f441ce45775e4780809b60f28f279128" is_pre_election: true
I20260806 22:49:33.797096 24441 raft_consensus.cc:2454] T aeb3c8b095c140fd8c785022413cc759 P f441ce45775e4780809b60f28f279128 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92c11be029d24732a8a680f7ee614722 in term 0.
I20260806 22:49:33.797371 24731 leader_election.cc:302] T aeb3c8b095c140fd8c785022413cc759 P 92c11be029d24732a8a680f7ee614722 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92c11be029d24732a8a680f7ee614722, f441ce45775e4780809b60f28f279128; no voters:
I20260806 22:49:33.797519 24943 raft_consensus.cc:2781] T aeb3c8b095c140fd8c785022413cc759 P 92c11be029d24732a8a680f7ee614722 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:49:33.797564 24943 raft_consensus.cc:482] T aeb3c8b095c140fd8c785022413cc759 P 92c11be029d24732a8a680f7ee614722 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:49:33.797587 24943 raft_consensus.cc:3037] T aeb3c8b095c140fd8c785022413cc759 P 92c11be029d24732a8a680f7ee614722 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:49:33.797847 24931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:33.791780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.67:56623 (local address 127.6.40.66:45347)
0806 22:49:33.791880 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:33.791883 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:33.792136 (+ 253us) server_negotiation.cc:407] Connection header received
0806 22:49:33.792248 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:33.792250 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:33.792263 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:33.792297 (+ 34us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:49:33.792556 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:33.794887 (+ 2331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:33.796876 (+ 1989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:33.797453 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:33.797534 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:33.797546 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:33.797738 (+ 192us) server_negotiation.cc:299] Negotiation successful
0806 22:49:33.797780 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 22:49:33.798103 24947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:33.791538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.66:45347 (local address 127.6.40.67:56623)
0806 22:49:33.792063 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:33.792075 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:33.792166 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:33.792370 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:33.792370 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:33.792372 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:49:33.792473 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:33.792474 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:33.795003 (+ 2529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:33.795003 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:33.796759 (+ 1756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:33.796762 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:33.797583 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:33.797584 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:33.797660 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:33.797663 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:49:33.797971 (+ 308us) client_negotiation.cc:240] Negotiation successful
0806 22:49:33.798015 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":107,"threads_started":1}
I20260806 22:49:33.798434 24614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeb3c8b095c140fd8c785022413cc759" candidate_uuid: "92c11be029d24732a8a680f7ee614722" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a7262b9033a4215aadce65d578d154e" is_pre_election: true
I20260806 22:49:33.798600 24614 raft_consensus.cc:2454] T aeb3c8b095c140fd8c785022413cc759 P 4a7262b9033a4215aadce65d578d154e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92c11be029d24732a8a680f7ee614722 in term 0.
I20260806 22:49:33.798573 24943 raft_consensus.cc:504] T aeb3c8b095c140fd8c785022413cc759 P 92c11be029d24732a8a680f7ee614722 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92c11be029d24732a8a680f7ee614722" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 40151 } } peers { permanent_uuid: "f441ce45775e4780809b60f28f279128" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 45817 } } peers { permanent_uuid: "4a7262b9033a4215aadce65d578d154e" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 45347 } }
I20260806 22:49:33.798717 24943 leader_election.cc:288] T aeb3c8b095c140fd8c785022413cc759 P 92c11be029d24732a8a680f7ee614722 [CANDIDATE]: Term 1 election: Requested vote from peers f441ce45775e4780809b60f28f279128 (127.6.40.65:45817), 4a7262b9033a4215aadce65d578d154e (127.6.40.66:45347)
I20260806 22:49:33.798877 24441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeb3c8b095c140fd8c785022413cc759" candidate_uuid: "92c11be029d24732a8a680f7ee614722" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f441ce45775e4780809b60f28f279128"
I20260806 22:49:33.798969 24441 raft_consensus.cc:3037] T aeb3c8b095c140fd8c785022413cc759 P f441ce45775e4780809b60f28f279128 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:49:33.799034 24614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeb3c8b095c140fd8c785022413cc759" candidate_uuid: "92c11be029d24732a8a680f7ee614722" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a7262b9033a4215aadce65d578d154e"
I20260806 22:49:33.799127 24614 raft_consensus.cc:3037] T aeb3c8b095c140fd8c785022413cc759 P 4a7262b9033a4215aadce65d578d154e [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:49:33.800006 24441 raft_consensus.cc:2454] T aeb3c8b095c140fd8c785022413cc759 P f441ce45775e4780809b60f28f279128 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92c11be029d24732a8a680f7ee614722 in term 1.
I20260806 22:49:33.800150 24614 raft_consensus.cc:2454] T aeb3c8b095c140fd8c785022413cc759 P 4a7262b9033a4215aadce65d578d154e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92c11be029d24732a8a680f7ee614722 in term 1.
I20260806 22:49:33.800215 24731 leader_election.cc:302] T aeb3c8b095c140fd8c785022413cc759 P 92c11be029d24732a8a680f7ee614722 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92c11be029d24732a8a680f7ee614722, f441ce45775e4780809b60f28f279128; no voters:
I20260806 22:49:33.800313 24943 raft_consensus.cc:2781] T aeb3c8b095c140fd8c785022413cc759 P 92c11be029d24732a8a680f7ee614722 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:49:33.800482 24943 raft_consensus.cc:686] T aeb3c8b095c140fd8c785022413cc759 P 92c11be029d24732a8a680f7ee614722 [term 1 LEADER]: Becoming Leader. State: Replica: 92c11be029d24732a8a680f7ee614722, State: Running, Role: LEADER
I20260806 22:49:33.800585 24943 consensus_queue.cc:237] T aeb3c8b095c140fd8c785022413cc759 P 92c11be029d24732a8a680f7ee614722 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92c11be029d24732a8a680f7ee614722" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 40151 } } peers { permanent_uuid: "f441ce45775e4780809b60f28f279128" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 45817 } } peers { permanent_uuid: "4a7262b9033a4215aadce65d578d154e" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 45347 } }
I20260806 22:49:33.801501 24212 catalog_manager.cc:5495] T aeb3c8b095c140fd8c785022413cc759 P 92c11be029d24732a8a680f7ee614722 reported cstate change: term changed from 0 to 1, leader changed from <none> to 92c11be029d24732a8a680f7ee614722 (127.6.40.67). New cstate: current_term: 1 leader_uuid: "92c11be029d24732a8a680f7ee614722" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92c11be029d24732a8a680f7ee614722" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 40151 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f441ce45775e4780809b60f28f279128" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 45817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a7262b9033a4215aadce65d578d154e" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 45347 } health_report { overall_health: UNKNOWN } } }
I20260806 22:49:33.817776 24932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:33.814712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35528 (local address 127.6.40.67:40151)
0806 22:49:33.814761 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:33.814764 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:33.814778 (+ 14us) server_negotiation.cc:407] Connection header received
0806 22:49:33.814881 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:33.814882 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:33.814913 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:33.814975 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:49:33.815147 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:33.815676 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:33.816783 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:33.817039 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:33.817131 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:33.817406 (+ 275us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:49:33.817509 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:49:33.817612 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:33.817659 (+ 47us) server_negotiation.cc:299] Negotiation successful
0806 22:49:33.817682 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 22:49:33.819818 24441 raft_consensus.cc:1261] T aeb3c8b095c140fd8c785022413cc759 P f441ce45775e4780809b60f28f279128 [term 1 FOLLOWER]: Refusing update from remote peer 92c11be029d24732a8a680f7ee614722: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:49:33.819831 24614 raft_consensus.cc:1261] T aeb3c8b095c140fd8c785022413cc759 P 4a7262b9033a4215aadce65d578d154e [term 1 FOLLOWER]: Refusing update from remote peer 92c11be029d24732a8a680f7ee614722: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:49:33.820194 24951 consensus_queue.cc:1035] T aeb3c8b095c140fd8c785022413cc759 P 92c11be029d24732a8a680f7ee614722 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f441ce45775e4780809b60f28f279128" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 45817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:49:33.820333 24943 consensus_queue.cc:1035] T aeb3c8b095c140fd8c785022413cc759 P 92c11be029d24732a8a680f7ee614722 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a7262b9033a4215aadce65d578d154e" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 45347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:49:33.823474 24959 mvcc.cc:205] Tried to move back new op lower bound from 7315687726363148288 to 7315687726287802368. Current Snapshot: MvccSnapshot[applied={T|T < 7315687726363148288}]
I20260806 22:49:33.823925 24958 mvcc.cc:205] Tried to move back new op lower bound from 7315687726363148288 to 7315687726287802368. Current Snapshot: MvccSnapshot[applied={T|T < 7315687726363148288}]
I20260806 22:49:33.823956 24960 mvcc.cc:205] Tried to move back new op lower bound from 7315687726363148288 to 7315687726287802368. Current Snapshot: MvccSnapshot[applied={T|T < 7315687726363148288}]
I20260806 22:49:33.827001 24212 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52822:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:49:33.827126 24212 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52822:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:49:33.828042 24212 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 83eb24ce61b740af80765c65d53e86fd: Sending DeleteTablet for 3 replicas of tablet aeb3c8b095c140fd8c785022413cc759
I20260806 22:49:33.828485 24410 tablet_service.cc:1514] Processing DeleteTablet for tablet aeb3c8b095c140fd8c785022413cc759 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:49:33 UTC) from {username='kudu', principal='kudu/127.6.40.126@KRBTEST.COM'} at 127.0.0.1:55984
I20260806 22:49:33.828639 24793 tablet_service.cc:1514] Processing DeleteTablet for tablet aeb3c8b095c140fd8c785022413cc759 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:49:33 UTC) from {username='kudu', principal='kudu/127.6.40.126@KRBTEST.COM'} at 127.0.0.1:35522
I20260806 22:49:33.828919 24972 tablet_replica.cc:330] T aeb3c8b095c140fd8c785022413cc759 P 92c11be029d24732a8a680f7ee614722: stopping tablet replica
I20260806 22:49:33.828946 24594 tablet_service.cc:1514] Processing DeleteTablet for tablet aeb3c8b095c140fd8c785022413cc759 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:49:33 UTC) from {username='kudu', principal='kudu/127.6.40.126@KRBTEST.COM'} at 127.0.0.1:52202
I20260806 22:49:33.829030 24972 raft_consensus.cc:2229] T aeb3c8b095c140fd8c785022413cc759 P 92c11be029d24732a8a680f7ee614722 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:49:33.829151 24972 raft_consensus.cc:2258] T aeb3c8b095c140fd8c785022413cc759 P 92c11be029d24732a8a680f7ee614722 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:49:33.829579 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 24270
I20260806 22:49:33.829602 24972 ts_tablet_manager.cc:1905] T aeb3c8b095c140fd8c785022413cc759 P 92c11be029d24732a8a680f7ee614722: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:49:33.832125 24972 ts_tablet_manager.cc:1918] T aeb3c8b095c140fd8c785022413cc759 P 92c11be029d24732a8a680f7ee614722: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:49:33.832221 24972 log.cc:1196] T aeb3c8b095c140fd8c785022413cc759 P 92c11be029d24732a8a680f7ee614722: Deleting WAL directory at /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ts-2/wal/wals/aeb3c8b095c140fd8c785022413cc759
I20260806 22:49:33.832578 24972 ts_tablet_manager.cc:1939] T aeb3c8b095c140fd8c785022413cc759 P 92c11be029d24732a8a680f7ee614722: Deleting consensus metadata
I20260806 22:49:33.837239 24185 catalog_manager.cc:4841] TS 92c11be029d24732a8a680f7ee614722 (127.6.40.67:40151): tablet aeb3c8b095c140fd8c785022413cc759 (table test-table [id=0fbbb627a99c4967a90d6604fd9999c0]) successfully deleted
I20260806 22:49:33.840950 24973 tablet_replica.cc:330] T aeb3c8b095c140fd8c785022413cc759 P 4a7262b9033a4215aadce65d578d154e: stopping tablet replica
I20260806 22:49:33.841114 24973 raft_consensus.cc:2229] T aeb3c8b095c140fd8c785022413cc759 P 4a7262b9033a4215aadce65d578d154e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:49:33.841270 24973 raft_consensus.cc:2258] T aeb3c8b095c140fd8c785022413cc759 P 4a7262b9033a4215aadce65d578d154e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:49:33.841943 24973 ts_tablet_manager.cc:1905] T aeb3c8b095c140fd8c785022413cc759 P 4a7262b9033a4215aadce65d578d154e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:49:33.844158 24973 ts_tablet_manager.cc:1918] T aeb3c8b095c140fd8c785022413cc759 P 4a7262b9033a4215aadce65d578d154e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:49:33.844309 24973 log.cc:1196] T aeb3c8b095c140fd8c785022413cc759 P 4a7262b9033a4215aadce65d578d154e: Deleting WAL directory at /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056405724609-6305-0/minicluster-data/ts-1/wal/wals/aeb3c8b095c140fd8c785022413cc759
I20260806 22:49:33.844672 24973 ts_tablet_manager.cc:1939] T aeb3c8b095c140fd8c785022413cc759 P 4a7262b9033a4215aadce65d578d154e: Deleting consensus metadata
I20260806 22:49:33.845221 24185 catalog_manager.cc:4841] TS 4a7262b9033a4215aadce65d578d154e (127.6.40.66:45347): tablet aeb3c8b095c140fd8c785022413cc759 (table test-table [id=0fbbb627a99c4967a90d6604fd9999c0]) successfully deleted
W20260806 22:49:33.855330 24184 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.40.65:45817 (error 108)
I20260806 22:49:33.856117 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 24512
I20260806 22:49:33.869509 24933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:33.869273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.65:45817 (local address 127.0.0.1:55992)
0806 22:49:33.869380 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:33.869445 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.65:45817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260806 22:49:33.869736 24184 catalog_manager.cc:4584] TS f441ce45775e4780809b60f28f279128 (127.6.40.65:45817): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet aeb3c8b095c140fd8c785022413cc759: Network error: Client connection negotiation failed: client connection to 127.6.40.65:45817: connect: Connection refused (error 111)
I20260806 22:49:33.895361 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 24704
I20260806 22:49:33.904796 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 24158
I20260806 22:49:33.926450 6305 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260806 22:49:34.272606 6305 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260806 22:49:34.272712 6305 mini_ranger.cc:66] Stopping Ranger...
I20260806 22:49:34.627282 6305 mini_ranger.cc:68] Stopped Ranger
2026-08-06 22:49:34.627 UTC [22818] LOG: received smart shutdown request
2026-08-06 22:49:34.631 UTC [22818] LOG: background worker "logical replication launcher" (PID 22832) exited with exit code 1
2026-08-06 22:49:34.632 UTC [22824] LOG: shutting down
2026-08-06 22:49:34.804 UTC [22818] LOG: database system is shut down
2026-08-06T22:49:34Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (51710 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056405724609-6305-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:49:34 dist-test-slave-zh2d krb5kdc[24990](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:49:34 dist-test-slave-zh2d krb5kdc[24990](info): set up 2 sockets
Aug 06 22:49:34 dist-test-slave-zh2d krb5kdc[24990](info): commencing operation
krb5kdc: starting...
W20260806 22:49:36.168543 6305 mini_kdc.cc:121] Time spent starting KDC: real 1.225s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056405724609-6305-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056405724609-6305-0/krb5kdc/test-user.keytab.
Aug 06 22:49:36 dist-test-slave-zh2d krb5kdc[24990](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056576, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:49:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:49:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.40.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.40.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.40.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab.
Entry for principal kudu/127.6.40.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab.
I20260806 22:49:36.300000 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056405724609-6305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056405724609-6305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056405724609-6305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.126:33885
--webserver_interface=127.6.40.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab
--principal=kudu/127.6.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:42205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.40.126:33885,127.6.40.125:34183,127.6.40.124:39985
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:49:36.385344 25006 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:49:36.385577 25006 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:49:36.385613 25006 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:49:36.386770 25006 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:49:36.386840 25006 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:49:36.386868 25006 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:49:36.386900 25006 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:49:36.386930 25006 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:49:36.389184 25006 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:42205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056405724609-6305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056405724609-6305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.40.126:33885,127.6.40.125:34183,127.6.40.124:39985
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab
--principal=kudu/127.6.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.126:33885
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25006
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056405724609-6305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:49:36.389456 25006 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:49:36.389705 25006 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 22:49:36.394892 25006 server_base.cc:1034] running on GCE node
W20260806 22:49:36.395040 25017 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:36.395071 25011 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:36.395092 25013 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:36.395516 25006 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:49:36.395845 25006 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:49:36.396992 25006 hybrid_clock.cc:648] HybridClock initialized: now 1786056576396971 us; error 39 us; skew 500 ppm
Aug 06 22:49:36 dist-test-slave-zh2d krb5kdc[24990](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056576, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:49:36.399986 25006 init.cc:377] Logged in from keytab as kudu/127.6.40.126@KRBTEST.COM (short username kudu)
I20260806 22:49:36.400419 25006 webserver.cc:460] Webserver started at http://127.6.40.126:35315/ using document root <none> and password file <none>
I20260806 22:49:36.400698 25006 fs_manager.cc:359] Metadata directory not provided
I20260806 22:49:36.400744 25006 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:49:36.400925 25006 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:49:36.402030 25006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056405724609-6305-0/minicluster-data/master-0/data/instance:
uuid: "c45a838d17b542ac9697f282b50e44e5"
format_stamp: "Formatted at 2026-08-06 22:49:36 on dist-test-slave-zh2d"
server_key: "ae9818571e2b17a0f5fdf23d5016b4e9"
server_key_iv: "76bf6b933a884432d4b160778c9adbf4"
server_key_version: "encryptionkey@0"
I20260806 22:49:36.402402 25006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056405724609-6305-0/minicluster-data/master-0/wal/instance:
uuid: "c45a838d17b542ac9697f282b50e44e5"
format_stamp: "Formatted at 2026-08-06 22:49:36 on dist-test-slave-zh2d"
server_key: "ae9818571e2b17a0f5fdf23d5016b4e9"
server_key_iv: "76bf6b933a884432d4b160778c9adbf4"
server_key_version: "encryptionkey@0"
I20260806 22:49:36.403756 25006 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:49:36.404448 25030 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:36.404624 25006 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:49:36.404727 25006 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056405724609-6305-0/minicluster-data/master-0/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056405724609-6305-0/minicluster-data/master-0/wal
uuid: "c45a838d17b542ac9697f282b50e44e5"
format_stamp: "Formatted at 2026-08-06 22:49:36 on dist-test-slave-zh2d"
server_key: "ae9818571e2b17a0f5fdf23d5016b4e9"
server_key_iv: "76bf6b933a884432d4b160778c9adbf4"
server_key_version: "encryptionkey@0"
I20260806 22:49:36.404805 25006 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056405724609-6305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056405724609-6305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056405724609-6305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:49:36.414606 25006 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:49:36.416441 25006 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:49:36.416599 25006 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:49:36.421244 25006 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.126:33885
I20260806 22:49:36.421291 25110 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.126:33885 every 8 connection(s)
I20260806 22:49:36.421654 25006 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
I20260806 22:49:36.422550 25111 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:49:36.424136 25111 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.40.126" port: 33885 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:49:36.425822 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 25006
I20260806 22:49:36.425937 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056405724609-6305-0/minicluster-data/master-0/wal/instance
I20260806 22:49:36.426147 6305 external_mini_cluster.cc:1428] Setting key 84b2327d34013d8adfd7d8177a3c9ec3
Aug 06 22:49:36 dist-test-slave-zh2d krb5kdc[24990](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056576, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.126@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
I20260806 22:49:36.434926 25114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:36.424911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41110 (local address 127.6.40.126:33885)
0806 22:49:36.425116 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:36.425125 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:36.425213 (+ 88us) server_negotiation.cc:407] Connection header received
0806 22:49:36.425723 (+ 510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:36.425725 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:36.425740 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:36.425816 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:36.426580 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:36.427396 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:36.428490 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:36.429034 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:36.429147 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:36.431561 (+ 2414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:36.431568 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:36.431569 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:36.431575 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:36.433223 (+ 1648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:36.433742 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:36.433745 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:36.433745 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:36.433828 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:36.434112 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:36.434113 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:36.434114 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:36.434321 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:36.434389 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:36.434642 (+ 253us) server_negotiation.cc:299] Negotiation successful
0806 22:49:36.434762 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":109,"threads_started":1}
I20260806 22:49:36.435042 25113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:36.424887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:33885 (local address 127.0.0.1:41110)
0806 22:49:36.425087 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:36.425117 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:36.425570 (+ 453us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:36.425862 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:36.425865 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:36.426247 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:49:36.426478 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:36.426484 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:36.427498 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:36.427499 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:36.428400 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:36.428402 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:36.429121 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:36.429122 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:36.429182 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:36.429706 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:49:36.429714 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:49:36.431464 (+ 1750us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:49:36.433317 (+ 1853us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:36.433326 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:36.433328 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:36.433622 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:36.433911 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:36.433912 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:36.433912 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:36.434026 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:36.434399 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:49:36.434408 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:49:36.434500 (+ 92us) client_negotiation.cc:769] Sending connection context
0806 22:49:36.434615 (+ 115us) client_negotiation.cc:240] Negotiation successful
0806 22:49:36.434879 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"mutex_wait_us":80,"thread_start_us":90,"threads_started":1}
WARNING: no policy specified for kudu/127.6.40.125@KRBTEST.COM; defaulting to no policy
I20260806 22:49:36.435854 25111 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.40.125" port: 34183 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:49:36.436234 25113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:36.436121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.125:34183 (local address 127.0.0.1:52998)
0806 22:49:36.436172 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:36.436216 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.125:34183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260806 22:49:36.436367 25033 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.40.125:34183: connect: Connection refused (error 111)
I20260806 22:49:36.436892 25113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:36.436765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.125:34183 (local address 127.0.0.1:53000)
0806 22:49:36.436808 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:36.436832 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.125:34183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260806 22:49:36.437008 25111 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.40.125:34183: Network error: Client connection negotiation failed: client connection to 127.6.40.125:34183: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.40.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.40.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.125.keytab.
Entry for principal kudu/127.6.40.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.125.keytab.
I20260806 22:49:36.456019 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056405724609-6305-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056405724609-6305-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056405724609-6305-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056405724609-6305-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.125:34183
--webserver_interface=127.6.40.125
--webserver_port=0
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.125.keytab
--principal=kudu/127.6.40.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:42205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.40.126:33885,127.6.40.125:34183,127.6.40.124:39985
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:49:36.486241 25111 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.40.125" port: 34183 } attempt: 1
I20260806 22:49:36.486843 25113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:36.486688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.125:34183 (local address 127.0.0.1:53006)
0806 22:49:36.486763 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:36.486811 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.125:34183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260806 22:49:36.487283 25113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:36.487200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.125:34183 (local address 127.0.0.1:53010)
0806 22:49:36.487241 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:36.487266 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.125:34183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260806 22:49:36.487389 25111 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.40.125:34183: Network error: Client connection negotiation failed: client connection to 127.6.40.125:34183: connect: Connection refused (error 111)
W20260806 22:49:36.541771 25119 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:49:36.542006 25119 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:49:36.542045 25119 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:49:36.543241 25119 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:49:36.543303 25119 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:49:36.543324 25119 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:49:36.543352 25119 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:49:36.543378 25119 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:49:36.544970 25119 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:42205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056405724609-6305-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056405724609-6305-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.6.40.126:33885,127.6.40.125:34183,127.6.40.124:39985
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.125.keytab
--principal=kudu/127.6.40.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.125:34183
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056405724609-6305-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.6.40.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25119
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056405724609-6305-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:49:36.545249 25119 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:49:36.545486 25119 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:49:36.550632 25128 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:36.550627 25129 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:36.550676 25119 server_base.cc:1034] running on GCE node
W20260806 22:49:36.550627 25131 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:36.551079 25119 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:49:36.551357 25119 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:49:36.552531 25119 hybrid_clock.cc:648] HybridClock initialized: now 1786056576552521 us; error 52 us; skew 500 ppm
Aug 06 22:49:36 dist-test-slave-zh2d krb5kdc[24990](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056576, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:49:36.555521 25111 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.40.125" port: 34183 } attempt: 2
I20260806 22:49:36.555586 25119 init.cc:377] Logged in from keytab as kudu/127.6.40.125@KRBTEST.COM (short username kudu)
I20260806 22:49:36.555919 25113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:36.555787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.125:34183 (local address 127.0.0.1:53020)
0806 22:49:36.555848 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:36.555875 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.125:34183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260806 22:49:36.556133 25119 webserver.cc:460] Webserver started at http://127.6.40.125:39441/ using document root <none> and password file <none>
I20260806 22:49:36.556231 25113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:36.556152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.125:34183 (local address 127.0.0.1:53024)
0806 22:49:36.556182 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:36.556206 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.125:34183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260806 22:49:36.556331 25111 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.40.125:34183: Network error: Client connection negotiation failed: client connection to 127.6.40.125:34183: connect: Connection refused (error 111)
I20260806 22:49:36.556428 25119 fs_manager.cc:359] Metadata directory not provided
I20260806 22:49:36.556512 25119 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:49:36.556651 25119 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:49:36.557765 25119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056405724609-6305-0/minicluster-data/master-1/data/instance:
uuid: "6ab2bc57725c4628adf19d0081ffe422"
format_stamp: "Formatted at 2026-08-06 22:49:36 on dist-test-slave-zh2d"
server_key: "1c2028b40582fd082de3bd8d847436fe"
server_key_iv: "eb6bc1848c477ce69f92ed30f8ed3938"
server_key_version: "encryptionkey@0"
I20260806 22:49:36.558140 25119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056405724609-6305-0/minicluster-data/master-1/wal/instance:
uuid: "6ab2bc57725c4628adf19d0081ffe422"
format_stamp: "Formatted at 2026-08-06 22:49:36 on dist-test-slave-zh2d"
server_key: "1c2028b40582fd082de3bd8d847436fe"
server_key_iv: "eb6bc1848c477ce69f92ed30f8ed3938"
server_key_version: "encryptionkey@0"
I20260806 22:49:36.559476 25119 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:49:36.560173 25140 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:36.560350 25119 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:49:36.560449 25119 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056405724609-6305-0/minicluster-data/master-1/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056405724609-6305-0/minicluster-data/master-1/wal
uuid: "6ab2bc57725c4628adf19d0081ffe422"
format_stamp: "Formatted at 2026-08-06 22:49:36 on dist-test-slave-zh2d"
server_key: "1c2028b40582fd082de3bd8d847436fe"
server_key_iv: "eb6bc1848c477ce69f92ed30f8ed3938"
server_key_version: "encryptionkey@0"
I20260806 22:49:36.560530 25119 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056405724609-6305-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056405724609-6305-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056405724609-6305-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:49:36.583834 25119 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:49:36.586059 25119 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:49:36.586246 25119 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:49:36.590646 25119 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.125:34183
I20260806 22:49:36.590689 25214 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.125:34183 every 8 connection(s)
I20260806 22:49:36.591100 25119 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056405724609-6305-0/minicluster-data/master-1/data/info.pb
I20260806 22:49:36.591643 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 25119
I20260806 22:49:36.591728 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056405724609-6305-0/minicluster-data/master-1/wal/instance
I20260806 22:49:36.591958 6305 external_mini_cluster.cc:1428] Setting key 360a029e2fa8d72207c997a7ae5e1cd4
I20260806 22:49:36.592118 25216 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:49:36.593840 25216 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.40.126" port: 33885 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 22:49:36 dist-test-slave-zh2d krb5kdc[24990](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056576, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.125@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.40.124@KRBTEST.COM; defaulting to no policy
I20260806 22:49:36.603615 25114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:36.594635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41116 (local address 127.6.40.126:33885)
0806 22:49:36.594721 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:36.594728 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:36.594917 (+ 189us) server_negotiation.cc:407] Connection header received
0806 22:49:36.595430 (+ 513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:36.595431 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:36.595444 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:36.595511 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:36.596186 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:36.596995 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:36.598068 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:36.598483 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:36.598570 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:36.600744 (+ 2174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:36.600746 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:36.600746 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:36.600754 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:36.602235 (+ 1481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:36.602722 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:36.602723 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:36.602723 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:36.602777 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:36.602997 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:36.602997 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:36.602997 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:36.603102 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:36.603153 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:36.603404 (+ 251us) server_negotiation.cc:299] Negotiation successful
0806 22:49:36.603553 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 22:49:36.603652 25220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:36.594601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:33885 (local address 127.0.0.1:41116)
0806 22:49:36.594794 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:36.594832 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:36.595322 (+ 490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:36.595597 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:36.595600 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:36.595888 (+ 288us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:49:36.596083 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:36.596088 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:36.597135 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:36.597137 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:36.597965 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:36.597968 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:36.598585 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:36.598586 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:36.598692 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:36.599242 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:49:36.599250 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:49:36.600640 (+ 1390us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:49:36.602413 (+ 1773us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:36.602420 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:36.602422 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:36.602655 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:36.602839 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:36.602840 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:36.602840 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:36.602941 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:36.603159 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:49:36.603164 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:49:36.603280 (+ 116us) client_negotiation.cc:769] Sending connection context
0806 22:49:36.603362 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 22:49:36.603483 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":87,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.40.124@KRBTEST.COM" created.
I20260806 22:49:36.604364 25216 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.40.125" port: 34183 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 22:49:36 dist-test-slave-zh2d krb5kdc[24990](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056576, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.125@KRBTEST.COM for kudu/127.6.40.125@KRBTEST.COM
I20260806 22:49:36.611776 25220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:36.604652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.125:34183 (local address 127.0.0.1:53034)
0806 22:49:36.604716 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:36.604726 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:36.604769 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:36.605061 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:36.605062 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:36.605207 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:49:36.605252 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:36.605253 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:36.606124 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:36.606125 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:36.606621 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:36.606622 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:36.607109 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:36.607110 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:36.607201 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:36.607635 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:49:36.607640 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:49:36.609019 (+ 1379us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:49:36.610688 (+ 1669us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:36.610688 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:36.610689 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:36.610854 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:36.611092 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:36.611092 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:36.611093 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:36.611148 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:36.611445 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:49:36.611446 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:49:36.611484 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 22:49:36.611562 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 22:49:36.611661 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260806 22:49:36.611945 25224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:36.604718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53034 (local address 127.6.40.125:34183)
0806 22:49:36.604900 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:36.604904 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:36.604930 (+ 26us) server_negotiation.cc:407] Connection header received
0806 22:49:36.604947 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:36.604949 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:36.604959 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:36.605058 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:36.605329 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:36.606033 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:36.606704 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:36.607006 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:36.607110 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:36.609115 (+ 2005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:36.609123 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:36.609124 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:36.609130 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:36.610587 (+ 1457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:36.610952 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:36.610955 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:36.610955 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:36.611018 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:36.611222 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:36.611223 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:36.611224 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:36.611373 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:36.611467 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:36.611593 (+ 126us) server_negotiation.cc:299] Negotiation successful
0806 22:49:36.611737 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":57,"server-negotiator.queue_time_us":153,"thread_start_us":87,"threads_started":1}
I20260806 22:49:36.612569 25216 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.40.124" port: 39985 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:49:36.612962 25220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:36.612804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.124:39985 (local address 127.0.0.1:43572)
0806 22:49:36.612880 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:36.612938 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.124:39985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
W20260806 22:49:36.613145 25145 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.40.124:39985: connect: Connection refused (error 111)
I20260806 22:49:36.613622 25220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:36.613536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.124:39985 (local address 127.0.0.1:43578)
0806 22:49:36.613580 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:36.613605 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.124:39985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260806 22:49:36.613741 25216 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.40.124:39985: Network error: Client connection negotiation failed: client connection to 127.6.40.124:39985: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.40.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.124.keytab.
Entry for principal kudu/127.6.40.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.124.keytab.
I20260806 22:49:36.621577 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056405724609-6305-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056405724609-6305-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056405724609-6305-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056405724609-6305-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.124:39985
--webserver_interface=127.6.40.124
--webserver_port=0
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.124.keytab
--principal=kudu/127.6.40.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:42205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.40.126:33885,127.6.40.125:34183,127.6.40.124:39985
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:49:36.647491 25111 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.40.125" port: 34183 } attempt: 3
Aug 06 22:49:36 dist-test-slave-zh2d krb5kdc[24990](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056576, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.126@KRBTEST.COM for kudu/127.6.40.125@KRBTEST.COM
I20260806 22:49:36.654831 25113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:36.647888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.125:34183 (local address 127.0.0.1:53040)
0806 22:49:36.647952 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:36.647966 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:36.648042 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:36.648202 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:36.648203 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:36.648373 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:49:36.648442 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:36.648444 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:36.649202 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:36.649208 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:36.649735 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:36.649736 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:36.650125 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:36.650126 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:36.650215 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:36.650725 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:49:36.650729 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:49:36.652129 (+ 1400us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:49:36.653906 (+ 1777us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:36.653907 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:36.653907 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:36.654078 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:36.654283 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:36.654284 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:36.654284 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:36.654334 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:36.654593 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:49:36.654594 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:49:36.654631 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 22:49:36.654683 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 22:49:36.654760 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260806 22:49:36.654906 25224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:36.647986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53040 (local address 127.6.40.125:34183)
0806 22:49:36.648059 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:36.648061 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:36.648073 (+ 12us) server_negotiation.cc:407] Connection header received
0806 22:49:36.648105 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:36.648106 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:36.648117 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:36.648193 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:36.648536 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:36.649106 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:36.649820 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:36.650042 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:36.650128 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:36.652225 (+ 2097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:36.652226 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:36.652227 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:36.652236 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:36.653822 (+ 1586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:36.654150 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:36.654152 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:36.654152 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:36.654216 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:36.654393 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:36.654394 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:36.654394 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:36.654527 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:36.654600 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:36.654727 (+ 127us) server_negotiation.cc:299] Negotiation successful
0806 22:49:36.654814 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 22:49:36.655333 25111 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.40.124" port: 39985 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:49:36.655721 25113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:36.655623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.124:39985 (local address 127.0.0.1:43594)
0806 22:49:36.655672 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:36.655701 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.124:39985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260806 22:49:36.656090 25113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:36.655993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.124:39985 (local address 127.0.0.1:43608)
0806 22:49:36.656040 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:36.656066 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.124:39985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260806 22:49:36.656179 25111 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.40.124:39985: Network error: Client connection negotiation failed: client connection to 127.6.40.124:39985: connect: Connection refused (error 111)
I20260806 22:49:36.663023 25216 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.40.124" port: 39985 } attempt: 1
I20260806 22:49:36.663414 25220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:36.663283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.124:39985 (local address 127.0.0.1:43612)
0806 22:49:36.663360 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:36.663388 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.124:39985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260806 22:49:36.663764 25220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:36.663643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.124:39985 (local address 127.0.0.1:43622)
0806 22:49:36.663711 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:36.663736 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.124:39985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260806 22:49:36.663879 25216 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.40.124:39985: Network error: Client connection negotiation failed: client connection to 127.6.40.124:39985: connect: Connection refused (error 111)
I20260806 22:49:36.687374 25111 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.40.124" port: 39985 } attempt: 1
I20260806 22:49:36.687935 25113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:36.687769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.124:39985 (local address 127.0.0.1:43630)
0806 22:49:36.687853 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:36.687902 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.124:39985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260806 22:49:36.688333 25113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:36.688223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.124:39985 (local address 127.0.0.1:43642)
0806 22:49:36.688271 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:36.688310 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.124:39985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260806 22:49:36.688434 25111 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.40.124:39985: Network error: Client connection negotiation failed: client connection to 127.6.40.124:39985: connect: Connection refused (error 111)
W20260806 22:49:36.709362 25230 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:49:36.709595 25230 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:49:36.709633 25230 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:49:36.710776 25230 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:49:36.710845 25230 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:49:36.710870 25230 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:49:36.710891 25230 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:49:36.710922 25230 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:49:36.712410 25230 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:42205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056405724609-6305-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056405724609-6305-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.6.40.126:33885,127.6.40.125:34183,127.6.40.124:39985
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.124.keytab
--principal=kudu/127.6.40.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.124:39985
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056405724609-6305-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.6.40.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25230
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056405724609-6305-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:49:36.712690 25230 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:49:36.712990 25230 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:49:36.717959 25239 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:36.717994 25238 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:36.717971 25243 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:36.718464 25230 server_base.cc:1034] running on GCE node
I20260806 22:49:36.718736 25230 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:49:36.719033 25230 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:49:36.720209 25230 hybrid_clock.cc:648] HybridClock initialized: now 1786056576720190 us; error 32 us; skew 500 ppm
Aug 06 22:49:36 dist-test-slave-zh2d krb5kdc[24990](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056576, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:49:36.722790 25230 init.cc:377] Logged in from keytab as kudu/127.6.40.124@KRBTEST.COM (short username kudu)
I20260806 22:49:36.723225 25230 webserver.cc:460] Webserver started at http://127.6.40.124:34601/ using document root <none> and password file <none>
I20260806 22:49:36.723464 25230 fs_manager.cc:359] Metadata directory not provided
I20260806 22:49:36.723534 25230 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:49:36.723688 25230 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:49:36.724768 25230 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056405724609-6305-0/minicluster-data/master-2/data/instance:
uuid: "8ed6ff85c9f34eefa1e6e9539787f5c0"
format_stamp: "Formatted at 2026-08-06 22:49:36 on dist-test-slave-zh2d"
server_key: "f749f2828b7947e66f8cd1a0cc95d5d1"
server_key_iv: "5de16240b6b6d5a37ce6cd36e22bd5cd"
server_key_version: "encryptionkey@0"
I20260806 22:49:36.725191 25230 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056405724609-6305-0/minicluster-data/master-2/wal/instance:
uuid: "8ed6ff85c9f34eefa1e6e9539787f5c0"
format_stamp: "Formatted at 2026-08-06 22:49:36 on dist-test-slave-zh2d"
server_key: "f749f2828b7947e66f8cd1a0cc95d5d1"
server_key_iv: "5de16240b6b6d5a37ce6cd36e22bd5cd"
server_key_version: "encryptionkey@0"
I20260806 22:49:36.726567 25230 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:49:36.727283 25251 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:36.727473 25230 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:49:36.727574 25230 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056405724609-6305-0/minicluster-data/master-2/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056405724609-6305-0/minicluster-data/master-2/wal
uuid: "8ed6ff85c9f34eefa1e6e9539787f5c0"
format_stamp: "Formatted at 2026-08-06 22:49:36 on dist-test-slave-zh2d"
server_key: "f749f2828b7947e66f8cd1a0cc95d5d1"
server_key_iv: "5de16240b6b6d5a37ce6cd36e22bd5cd"
server_key_version: "encryptionkey@0"
I20260806 22:49:36.727658 25230 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056405724609-6305-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056405724609-6305-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056405724609-6305-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:49:36.732034 25216 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.40.124" port: 39985 } attempt: 2
I20260806 22:49:36.732451 25220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:36.732322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.124:39985 (local address 127.0.0.1:43656)
0806 22:49:36.732383 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:36.732422 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.124:39985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260806 22:49:36.732828 25220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:36.732727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.124:39985 (local address 127.0.0.1:43668)
0806 22:49:36.732772 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:36.732796 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.124:39985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260806 22:49:36.732945 25216 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.40.124:39985: Network error: Client connection negotiation failed: client connection to 127.6.40.124:39985: connect: Connection refused (error 111)
I20260806 22:49:36.743435 25230 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:49:36.745886 25230 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:49:36.746064 25230 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:49:36.750445 25230 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.124:39985
I20260806 22:49:36.750474 25329 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.124:39985 every 8 connection(s)
I20260806 22:49:36.750823 25230 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056405724609-6305-0/minicluster-data/master-2/data/info.pb
I20260806 22:49:36.751709 25330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:49:36.753439 25330 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.40.126" port: 33885 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:49:36.757110 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 25230
I20260806 22:49:36.757223 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056405724609-6305-0/minicluster-data/master-2/wal/instance
I20260806 22:49:36.757454 6305 external_mini_cluster.cc:1428] Setting key dd63d8a8a1536dcc45a6fb8ae6bffffb
Aug 06 22:49:36 dist-test-slave-zh2d krb5kdc[24990](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056576, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.124@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
I20260806 22:49:36.763579 25111 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.40.124" port: 39985 } attempt: 2
Aug 06 22:49:36 dist-test-slave-zh2d krb5kdc[24990](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056576, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.40.124@KRBTEST.COM
I20260806 22:49:36.764036 25332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:36.754123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:33885 (local address 127.0.0.1:41130)
0806 22:49:36.754311 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:36.754354 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:36.754809 (+ 455us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:36.755066 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:36.755070 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:36.755313 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:49:36.755505 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:36.755509 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:36.756145 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:36.756146 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:36.757043 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:36.757046 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:36.757551 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:36.757553 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:36.757716 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:36.758357 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:49:36.758367 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:49:36.760224 (+ 1857us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:49:36.762200 (+ 1976us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:36.762226 (+ 26us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:36.762228 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:36.762592 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:36.762795 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:36.762797 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:36.762797 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:36.762979 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:36.763297 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:49:36.763305 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:49:36.763481 (+ 176us) client_negotiation.cc:769] Sending connection context
0806 22:49:36.763597 (+ 116us) client_negotiation.cc:240] Negotiation successful
0806 22:49:36.763764 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":109,"threads_started":1}
I20260806 22:49:36.764076 25114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:36.754167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41130 (local address 127.6.40.126:33885)
0806 22:49:36.754237 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:36.754238 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:36.754466 (+ 228us) server_negotiation.cc:407] Connection header received
0806 22:49:36.754899 (+ 433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:36.754901 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:36.754911 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:36.754988 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:36.755594 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:36.756068 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:36.757126 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:36.757433 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:36.757515 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:36.760325 (+ 2810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:36.760327 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:36.760327 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:36.760335 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:36.762004 (+ 1669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:36.762664 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:36.762665 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:36.762665 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:36.762720 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:36.763068 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:36.763068 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:36.763069 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:36.763210 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:36.763276 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:36.763638 (+ 362us) server_negotiation.cc:299] Negotiation successful
0806 22:49:36.763983 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 22:49:36.764617 25330 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.40.125" port: 34183 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 22:49:36 dist-test-slave-zh2d krb5kdc[24990](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056576, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.126@KRBTEST.COM for kudu/127.6.40.124@KRBTEST.COM
Aug 06 22:49:36 dist-test-slave-zh2d krb5kdc[24990](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056576, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.124@KRBTEST.COM for kudu/127.6.40.125@KRBTEST.COM
I20260806 22:49:36.769176 25335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:36.757901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43682 (local address 127.6.40.124:39985)
0806 22:49:36.758105 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:36.758110 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:36.758210 (+ 100us) server_negotiation.cc:407] Connection header received
0806 22:49:36.758243 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:36.758245 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:36.758273 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:36.758345 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:36.759149 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:36.760255 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:36.761234 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:36.761782 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:36.761859 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:36.764882 (+ 3023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:36.764890 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:36.764890 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:36.764897 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:36.767583 (+ 2686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:36.767970 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:36.767973 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:36.767974 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:36.768040 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:36.768280 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:36.768281 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:36.768281 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:36.768465 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:36.768613 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:36.768974 (+ 361us) server_negotiation.cc:299] Negotiation successful
0806 22:49:36.769069 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":107,"threads_started":1}
I20260806 22:49:36.772251 25113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:36.763821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.124:39985 (local address 127.0.0.1:43694)
0806 22:49:36.763871 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:36.763881 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:36.763913 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:36.764319 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:36.764321 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:36.764476 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:49:36.764521 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:36.764523 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:36.765449 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:36.765450 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:36.765942 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:36.765943 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:36.766474 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:36.766476 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:36.766535 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:36.766885 (+ 350us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:49:36.766890 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:49:36.768297 (+ 1407us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:49:36.771000 (+ 2703us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:36.771001 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:36.771002 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:36.771249 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:36.771535 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:36.771537 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:36.771537 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:36.771601 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:36.771894 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:49:36.771895 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:49:36.771954 (+ 59us) client_negotiation.cc:769] Sending connection context
0806 22:49:36.772042 (+ 88us) client_negotiation.cc:240] Negotiation successful
0806 22:49:36.772149 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260806 22:49:36.772313 25336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:36.764014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43694 (local address 127.6.40.124:39985)
0806 22:49:36.764160 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:36.764162 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:36.764170 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:49:36.764190 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:36.764191 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:36.764203 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:36.764596 (+ 393us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:36.764633 (+ 37us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:36.765370 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:36.766049 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:36.766374 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:36.766470 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:36.768407 (+ 1937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:36.768409 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:36.768409 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:36.768416 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:36.770816 (+ 2400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:36.771336 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:36.771337 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:36.771337 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:36.771443 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:36.771680 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:36.771681 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:36.771681 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:36.771809 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:36.771889 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:36.772096 (+ 207us) server_negotiation.cc:299] Negotiation successful
0806 22:49:36.772179 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":84,"threads_started":1}
I20260806 22:49:36.772432 25332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:36.764941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.125:34183 (local address 127.0.0.1:53054)
0806 22:49:36.764982 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:36.764992 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:36.765041 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:36.765213 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:36.765214 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:36.765350 (+ 136us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:49:36.765403 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:36.765405 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:36.766015 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:36.766016 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:36.766584 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:36.766587 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:36.767069 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:36.767070 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:36.767150 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:36.767554 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:49:36.767565 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:49:36.769074 (+ 1509us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:49:36.771336 (+ 2262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:36.771337 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:36.771337 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:36.771584 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:36.771798 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:36.771799 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:36.771799 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:36.771854 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:36.772164 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:49:36.772165 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:49:36.772200 (+ 35us) client_negotiation.cc:769] Sending connection context
0806 22:49:36.772261 (+ 61us) client_negotiation.cc:240] Negotiation successful
0806 22:49:36.772341 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260806 22:49:36.772748 25224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:36.764953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53054 (local address 127.6.40.125:34183)
0806 22:49:36.765001 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:36.765003 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:36.765028 (+ 25us) server_negotiation.cc:407] Connection header received
0806 22:49:36.765105 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:36.765105 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:36.765115 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:36.765166 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:36.765492 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:36.765923 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:36.766676 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:36.766960 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:36.767068 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:36.769168 (+ 2100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:36.769170 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:36.769171 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:36.769178 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:36.771251 (+ 2073us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:36.771672 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:36.771674 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:36.771674 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:36.771729 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:36.771922 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:36.771923 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:36.771923 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:36.772075 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:36.772136 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:36.772606 (+ 470us) server_negotiation.cc:299] Negotiation successful
0806 22:49:36.772687 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 22:49:36.774176 25111 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c45a838d17b542ac9697f282b50e44e5: Bootstrap starting.
I20260806 22:49:36.774813 25111 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c45a838d17b542ac9697f282b50e44e5: Neither blocks nor log segments found. Creating new log.
I20260806 22:49:36.775084 25111 log.cc:824] T 00000000000000000000000000000000 P c45a838d17b542ac9697f282b50e44e5: Log is configured to *not* fsync() on all Append() calls
I20260806 22:49:36.775374 25330 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.40.124" port: 39985 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:49:36.775995 25111 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c45a838d17b542ac9697f282b50e44e5: No bootstrap required, opened a new log
Aug 06 22:49:36 dist-test-slave-zh2d krb5kdc[24990](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056576, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
I20260806 22:49:36.777997 25111 raft_consensus.cc:348] T 00000000000000000000000000000000 P c45a838d17b542ac9697f282b50e44e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c45a838d17b542ac9697f282b50e44e5" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 33885 } } peers { permanent_uuid: "6ab2bc57725c4628adf19d0081ffe422" member_type: VOTER last_known_addr { host: "127.6.40.125" port: 34183 } } peers { permanent_uuid: "8ed6ff85c9f34eefa1e6e9539787f5c0" member_type: VOTER last_known_addr { host: "127.6.40.124" port: 39985 } }
I20260806 22:49:36.778139 25111 raft_consensus.cc:374] T 00000000000000000000000000000000 P c45a838d17b542ac9697f282b50e44e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:49:36.778177 25111 raft_consensus.cc:729] T 00000000000000000000000000000000 P c45a838d17b542ac9697f282b50e44e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c45a838d17b542ac9697f282b50e44e5, State: Initialized, Role: FOLLOWER
I20260806 22:49:36.778386 25111 consensus_queue.cc:260] T 00000000000000000000000000000000 P c45a838d17b542ac9697f282b50e44e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c45a838d17b542ac9697f282b50e44e5" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 33885 } } peers { permanent_uuid: "6ab2bc57725c4628adf19d0081ffe422" member_type: VOTER last_known_addr { host: "127.6.40.125" port: 34183 } } peers { permanent_uuid: "8ed6ff85c9f34eefa1e6e9539787f5c0" member_type: VOTER last_known_addr { host: "127.6.40.124" port: 39985 } }
I20260806 22:49:36.778779 25111 sys_catalog.cc:564] T 00000000000000000000000000000000 P c45a838d17b542ac9697f282b50e44e5 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:49:36.778945 25339 sys_catalog.cc:455] T 00000000000000000000000000000000 P c45a838d17b542ac9697f282b50e44e5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c45a838d17b542ac9697f282b50e44e5" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 33885 } } peers { permanent_uuid: "6ab2bc57725c4628adf19d0081ffe422" member_type: VOTER last_known_addr { host: "127.6.40.125" port: 34183 } } peers { permanent_uuid: "8ed6ff85c9f34eefa1e6e9539787f5c0" member_type: VOTER last_known_addr { host: "127.6.40.124" port: 39985 } } }
I20260806 22:49:36.779142 25339 sys_catalog.cc:458] T 00000000000000000000000000000000 P c45a838d17b542ac9697f282b50e44e5 [sys.catalog]: This master's current role is: FOLLOWER
Aug 06 22:49:36 dist-test-slave-zh2d krb5kdc[24990](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056576, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.124@KRBTEST.COM for kudu/127.6.40.124@KRBTEST.COM
I20260806 22:49:36.780395 25114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:36.770892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41140 (local address 127.6.40.126:33885)
0806 22:49:36.770933 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:36.770935 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:36.771435 (+ 500us) server_negotiation.cc:407] Connection header received
0806 22:49:36.771496 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:36.771497 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:36.771508 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:36.771567 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:36.772183 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:36.773076 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:36.773862 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:36.774090 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:36.774184 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:36.777056 (+ 2872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:36.777058 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:36.777058 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:36.777067 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:36.778779 (+ 1712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:36.779327 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:36.779329 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:36.779329 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:36.779406 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:36.779653 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:36.779654 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:36.779654 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:36.779800 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:36.779928 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:36.780181 (+ 253us) server_negotiation.cc:299] Negotiation successful
0806 22:49:36.780261 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
W20260806 22:49:36.780750 25353 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c45a838d17b542ac9697f282b50e44e5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:49:36.780802 25353 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:49:36.783735 25332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:36.775654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.124:39985 (local address 127.0.0.1:43710)
0806 22:49:36.775695 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:36.775704 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:36.775740 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:36.775896 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:36.775897 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:36.776056 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:49:36.776096 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:36.776098 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:36.777186 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:36.777187 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:36.777891 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:36.777894 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:36.778412 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:36.778413 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:36.778466 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:36.778859 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:49:36.778875 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:49:36.780492 (+ 1617us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:49:36.782680 (+ 2188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:36.782681 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:36.782681 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:36.782921 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:36.783122 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:36.783123 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:36.783123 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:36.783178 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:36.783474 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:49:36.783475 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:49:36.783509 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 22:49:36.783553 (+ 44us) client_negotiation.cc:240] Negotiation successful
0806 22:49:36.783639 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260806 22:49:36.784202 25336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:36.775700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43710 (local address 127.6.40.124:39985)
0806 22:49:36.775741 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:36.775743 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:36.775782 (+ 39us) server_negotiation.cc:407] Connection header received
0806 22:49:36.775807 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:36.775808 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:36.775819 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:36.775880 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:36.776199 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:36.777092 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:36.777987 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:36.778308 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:36.778607 (+ 299us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:36.780585 (+ 1978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:36.780586 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:36.780587 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:36.780593 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:36.782581 (+ 1988us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:36.783002 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:36.783002 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:36.783003 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:36.783056 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:36.783241 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:36.783241 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:36.783242 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:36.783402 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:36.783983 (+ 581us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:36.784026 (+ 43us) server_negotiation.cc:299] Negotiation successful
0806 22:49:36.784105 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 22:49:36.782172 25006 master_runner.cc:186] Error getting master registration for 127.6.40.125:34183: OK, instance_id { permanent_uuid: "6ab2bc57725c4628adf19d0081ffe422" instance_seqno: 1786056576589302 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 22:49:36.785454 25006 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 22:49:36.785842 25330 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ed6ff85c9f34eefa1e6e9539787f5c0: Bootstrap starting.
I20260806 22:49:36.786492 25330 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8ed6ff85c9f34eefa1e6e9539787f5c0: Neither blocks nor log segments found. Creating new log.
I20260806 22:49:36.786810 25330 log.cc:824] T 00000000000000000000000000000000 P 8ed6ff85c9f34eefa1e6e9539787f5c0: Log is configured to *not* fsync() on all Append() calls
Aug 06 22:49:36 dist-test-slave-zh2d krb5kdc[24990](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056576, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.40.125@KRBTEST.COM
I20260806 22:49:36.787624 25330 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ed6ff85c9f34eefa1e6e9539787f5c0: No bootstrap required, opened a new log
I20260806 22:49:36.789739 25330 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8ed6ff85c9f34eefa1e6e9539787f5c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c45a838d17b542ac9697f282b50e44e5" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 33885 } } peers { permanent_uuid: "6ab2bc57725c4628adf19d0081ffe422" member_type: VOTER last_known_addr { host: "127.6.40.125" port: 34183 } } peers { permanent_uuid: "8ed6ff85c9f34eefa1e6e9539787f5c0" member_type: VOTER last_known_addr { host: "127.6.40.124" port: 39985 } }
I20260806 22:49:36.789901 25330 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8ed6ff85c9f34eefa1e6e9539787f5c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:49:36.789988 25330 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8ed6ff85c9f34eefa1e6e9539787f5c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ed6ff85c9f34eefa1e6e9539787f5c0, State: Initialized, Role: FOLLOWER
I20260806 22:49:36.790197 25330 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8ed6ff85c9f34eefa1e6e9539787f5c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c45a838d17b542ac9697f282b50e44e5" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 33885 } } peers { permanent_uuid: "6ab2bc57725c4628adf19d0081ffe422" member_type: VOTER last_known_addr { host: "127.6.40.125" port: 34183 } } peers { permanent_uuid: "8ed6ff85c9f34eefa1e6e9539787f5c0" member_type: VOTER last_known_addr { host: "127.6.40.124" port: 39985 } }
I20260806 22:49:36.790556 25330 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8ed6ff85c9f34eefa1e6e9539787f5c0 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:49:36.790510 25356 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ed6ff85c9f34eefa1e6e9539787f5c0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c45a838d17b542ac9697f282b50e44e5" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 33885 } } peers { permanent_uuid: "6ab2bc57725c4628adf19d0081ffe422" member_type: VOTER last_known_addr { host: "127.6.40.125" port: 34183 } } peers { permanent_uuid: "8ed6ff85c9f34eefa1e6e9539787f5c0" member_type: VOTER last_known_addr { host: "127.6.40.124" port: 39985 } } }
I20260806 22:49:36.790738 25356 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ed6ff85c9f34eefa1e6e9539787f5c0 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:49:36.791746 25224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:36.781823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53060 (local address 127.6.40.125:34183)
0806 22:49:36.781866 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:36.781868 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:36.782029 (+ 161us) server_negotiation.cc:407] Connection header received
0806 22:49:36.782140 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:36.782140 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:36.782151 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:36.782184 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:36.782924 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:36.783717 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:36.784965 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:36.785187 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:36.785268 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:36.787791 (+ 2523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:36.787793 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:36.787793 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:36.787801 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:36.789458 (+ 1657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:36.789858 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:36.789860 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:36.789860 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:36.789938 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:36.790166 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:36.790167 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:36.790167 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:36.790312 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:36.791097 (+ 785us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:36.791310 (+ 213us) server_negotiation.cc:299] Negotiation successful
0806 22:49:36.791395 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 22:49:36.796558 6305 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260806 22:49:36.793498 25370 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8ed6ff85c9f34eefa1e6e9539787f5c0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:49:36.798310 25370 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:49:36.797104 25230 master_runner.cc:186] Error getting master registration for 127.6.40.125:34183: OK, instance_id { permanent_uuid: "6ab2bc57725c4628adf19d0081ffe422" instance_seqno: 1786056576589302 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 22:49:36.799422 25230 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 22:49:36.809578 25224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:36.801414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53064 (local address 127.6.40.125:34183)
0806 22:49:36.801456 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:36.801458 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:36.801697 (+ 239us) server_negotiation.cc:407] Connection header received
0806 22:49:36.801804 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:36.801805 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:36.801816 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:36.801873 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:36.802646 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:36.803381 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:36.804435 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:36.804738 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:36.804778 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:36.806452 (+ 1674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:36.806454 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:36.806454 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:36.806461 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:36.808168 (+ 1707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:36.808665 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:36.808667 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:36.808667 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:36.808723 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:36.808954 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:36.808955 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:36.808955 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:36.809085 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:36.809173 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:36.809385 (+ 212us) server_negotiation.cc:299] Negotiation successful
0806 22:49:36.809465 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 22:49:36.810722 25114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:36.801254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41154 (local address 127.6.40.126:33885)
0806 22:49:36.801303 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:36.801305 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:36.801590 (+ 285us) server_negotiation.cc:407] Connection header received
0806 22:49:36.801896 (+ 306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:36.801897 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:36.801908 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:36.801942 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:36.802747 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:36.803581 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:36.804711 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:36.805075 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:36.805132 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:36.806765 (+ 1633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:36.806767 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:36.806767 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:36.806774 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:36.808630 (+ 1856us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:36.809116 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:36.809117 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:36.809118 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:36.809177 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:36.809415 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:36.809416 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:36.809416 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:36.809552 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:36.809620 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:36.810525 (+ 905us) server_negotiation.cc:299] Negotiation successful
0806 22:49:36.810610 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 22:49:36.817196 25336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:36.801604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43724 (local address 127.6.40.124:39985)
0806 22:49:36.801652 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:36.801654 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:36.810050 (+ 8396us) server_negotiation.cc:407] Connection header received
0806 22:49:36.810146 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:36.810147 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:36.810157 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:36.810187 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:36.810727 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:36.811439 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:36.812421 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:36.812754 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:36.812795 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:36.814253 (+ 1458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:36.814254 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:36.814255 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:36.814262 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:36.815924 (+ 1662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:36.816308 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:36.816309 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:36.816309 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:36.816362 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:36.816570 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:36.816570 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:36.816571 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:36.816691 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:36.816773 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:36.817022 (+ 249us) server_negotiation.cc:299] Negotiation successful
0806 22:49:36.817106 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 22:49:36.824079 25216 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.40.124" port: 39985 } attempt: 3
Aug 06 22:49:36 dist-test-slave-zh2d krb5kdc[24990](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056576, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.125@KRBTEST.COM for kudu/127.6.40.124@KRBTEST.COM
I20260806 22:49:36.833142 25220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:36.824343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.124:39985 (local address 127.0.0.1:43726)
0806 22:49:36.824402 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:36.824413 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:36.824451 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:36.824718 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:36.824720 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:36.824961 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:49:36.825009 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:36.825011 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:36.825911 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:36.825912 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:36.826823 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:36.826826 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:36.827320 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:36.827322 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:36.827397 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:36.827871 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:49:36.827877 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:49:36.829817 (+ 1940us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:49:36.831917 (+ 2100us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:36.831918 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:36.831919 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:36.832143 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:36.832374 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:36.832374 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:36.832375 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:36.832441 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:36.832738 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:49:36.832739 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:49:36.832785 (+ 46us) client_negotiation.cc:769] Sending connection context
0806 22:49:36.832927 (+ 142us) client_negotiation.cc:240] Negotiation successful
0806 22:49:36.833027 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260806 22:49:36.833161 25336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:36.824532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43726 (local address 127.6.40.124:39985)
0806 22:49:36.824591 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:36.824593 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:36.824602 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:49:36.824622 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:36.824623 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:36.824632 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:36.824694 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:36.825090 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:36.825812 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:36.826918 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:36.827230 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:36.827305 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:36.829941 (+ 2636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:36.829942 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:36.829942 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:36.829949 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:36.831830 (+ 1881us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:36.832228 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:36.832230 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:36.832230 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:36.832296 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:36.832515 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:36.832516 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:36.832516 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:36.832663 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:36.832739 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:36.832960 (+ 221us) server_negotiation.cc:299] Negotiation successful
0806 22:49:36.833060 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 22:49:36.835278 25216 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ab2bc57725c4628adf19d0081ffe422: Bootstrap starting.
I20260806 22:49:36.836079 25216 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6ab2bc57725c4628adf19d0081ffe422: Neither blocks nor log segments found. Creating new log.
I20260806 22:49:36.836408 25216 log.cc:824] T 00000000000000000000000000000000 P 6ab2bc57725c4628adf19d0081ffe422: Log is configured to *not* fsync() on all Append() calls
I20260806 22:49:36.841653 25216 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ab2bc57725c4628adf19d0081ffe422: No bootstrap required, opened a new log
I20260806 22:49:36.843772 25216 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6ab2bc57725c4628adf19d0081ffe422 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c45a838d17b542ac9697f282b50e44e5" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 33885 } } peers { permanent_uuid: "6ab2bc57725c4628adf19d0081ffe422" member_type: VOTER last_known_addr { host: "127.6.40.125" port: 34183 } } peers { permanent_uuid: "8ed6ff85c9f34eefa1e6e9539787f5c0" member_type: VOTER last_known_addr { host: "127.6.40.124" port: 39985 } }
I20260806 22:49:36.843948 25216 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6ab2bc57725c4628adf19d0081ffe422 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:49:36.844007 25216 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6ab2bc57725c4628adf19d0081ffe422 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ab2bc57725c4628adf19d0081ffe422, State: Initialized, Role: FOLLOWER
I20260806 22:49:36.844189 25216 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6ab2bc57725c4628adf19d0081ffe422 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c45a838d17b542ac9697f282b50e44e5" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 33885 } } peers { permanent_uuid: "6ab2bc57725c4628adf19d0081ffe422" member_type: VOTER last_known_addr { host: "127.6.40.125" port: 34183 } } peers { permanent_uuid: "8ed6ff85c9f34eefa1e6e9539787f5c0" member_type: VOTER last_known_addr { host: "127.6.40.124" port: 39985 } }
I20260806 22:49:36.844606 25216 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6ab2bc57725c4628adf19d0081ffe422 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:49:36.844574 25383 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ab2bc57725c4628adf19d0081ffe422 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c45a838d17b542ac9697f282b50e44e5" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 33885 } } peers { permanent_uuid: "6ab2bc57725c4628adf19d0081ffe422" member_type: VOTER last_known_addr { host: "127.6.40.125" port: 34183 } } peers { permanent_uuid: "8ed6ff85c9f34eefa1e6e9539787f5c0" member_type: VOTER last_known_addr { host: "127.6.40.124" port: 39985 } } }
I20260806 22:49:36.844861 25383 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ab2bc57725c4628adf19d0081ffe422 [sys.catalog]: This master's current role is: FOLLOWER
W20260806 22:49:36.846611 25400 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6ab2bc57725c4628adf19d0081ffe422: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:49:36.846729 25400 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:49:36.848384 25119 master_runner.cc:263] No leader master found from master 6ab2bc57725c4628adf19d0081ffe422
I20260806 22:49:36.848433 25119 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260806 22:49:36.917446 25042 tablet.cc:2369] T 00000000000000000000000000000000 P c45a838d17b542ac9697f282b50e44e5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:49:36.984309 25383 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6ab2bc57725c4628adf19d0081ffe422 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:49:36.984458 25383 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6ab2bc57725c4628adf19d0081ffe422 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c45a838d17b542ac9697f282b50e44e5" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 33885 } } peers { permanent_uuid: "6ab2bc57725c4628adf19d0081ffe422" member_type: VOTER last_known_addr { host: "127.6.40.125" port: 34183 } } peers { permanent_uuid: "8ed6ff85c9f34eefa1e6e9539787f5c0" member_type: VOTER last_known_addr { host: "127.6.40.124" port: 39985 } }
I20260806 22:49:36.984766 25383 leader_election.cc:288] T 00000000000000000000000000000000 P 6ab2bc57725c4628adf19d0081ffe422 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c45a838d17b542ac9697f282b50e44e5 (127.6.40.126:33885), 8ed6ff85c9f34eefa1e6e9539787f5c0 (127.6.40.124:39985)
I20260806 22:49:36.985283 25076 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6ab2bc57725c4628adf19d0081ffe422" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c45a838d17b542ac9697f282b50e44e5" is_pre_election: true
I20260806 22:49:36.985466 25076 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c45a838d17b542ac9697f282b50e44e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ab2bc57725c4628adf19d0081ffe422 in term 0.
I20260806 22:49:36.985709 25142 leader_election.cc:302] T 00000000000000000000000000000000 P 6ab2bc57725c4628adf19d0081ffe422 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ab2bc57725c4628adf19d0081ffe422, c45a838d17b542ac9697f282b50e44e5; no voters:
I20260806 22:49:36.985819 25291 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6ab2bc57725c4628adf19d0081ffe422" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ed6ff85c9f34eefa1e6e9539787f5c0" is_pre_election: true
I20260806 22:49:36.985931 25383 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6ab2bc57725c4628adf19d0081ffe422 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:49:36.985983 25383 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6ab2bc57725c4628adf19d0081ffe422 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:49:36.985970 25291 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8ed6ff85c9f34eefa1e6e9539787f5c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ab2bc57725c4628adf19d0081ffe422 in term 0.
I20260806 22:49:36.986007 25383 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6ab2bc57725c4628adf19d0081ffe422 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:49:36.986815 25383 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6ab2bc57725c4628adf19d0081ffe422 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c45a838d17b542ac9697f282b50e44e5" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 33885 } } peers { permanent_uuid: "6ab2bc57725c4628adf19d0081ffe422" member_type: VOTER last_known_addr { host: "127.6.40.125" port: 34183 } } peers { permanent_uuid: "8ed6ff85c9f34eefa1e6e9539787f5c0" member_type: VOTER last_known_addr { host: "127.6.40.124" port: 39985 } }
I20260806 22:49:36.986966 25383 leader_election.cc:288] T 00000000000000000000000000000000 P 6ab2bc57725c4628adf19d0081ffe422 [CANDIDATE]: Term 1 election: Requested vote from peers c45a838d17b542ac9697f282b50e44e5 (127.6.40.126:33885), 8ed6ff85c9f34eefa1e6e9539787f5c0 (127.6.40.124:39985)
I20260806 22:49:36.987177 25291 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6ab2bc57725c4628adf19d0081ffe422" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ed6ff85c9f34eefa1e6e9539787f5c0"
I20260806 22:49:36.987176 25076 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6ab2bc57725c4628adf19d0081ffe422" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c45a838d17b542ac9697f282b50e44e5"
I20260806 22:49:36.987251 25291 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8ed6ff85c9f34eefa1e6e9539787f5c0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:49:36.987252 25076 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c45a838d17b542ac9697f282b50e44e5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:49:36.988092 25291 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8ed6ff85c9f34eefa1e6e9539787f5c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ab2bc57725c4628adf19d0081ffe422 in term 1.
I20260806 22:49:36.988169 25076 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c45a838d17b542ac9697f282b50e44e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ab2bc57725c4628adf19d0081ffe422 in term 1.
I20260806 22:49:36.988355 25145 leader_election.cc:302] T 00000000000000000000000000000000 P 6ab2bc57725c4628adf19d0081ffe422 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ab2bc57725c4628adf19d0081ffe422, 8ed6ff85c9f34eefa1e6e9539787f5c0; no voters:
I20260806 22:49:36.988464 25383 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6ab2bc57725c4628adf19d0081ffe422 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:49:36.988663 25383 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6ab2bc57725c4628adf19d0081ffe422 [term 1 LEADER]: Becoming Leader. State: Replica: 6ab2bc57725c4628adf19d0081ffe422, State: Running, Role: LEADER
I20260806 22:49:36.988756 25383 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6ab2bc57725c4628adf19d0081ffe422 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c45a838d17b542ac9697f282b50e44e5" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 33885 } } peers { permanent_uuid: "6ab2bc57725c4628adf19d0081ffe422" member_type: VOTER last_known_addr { host: "127.6.40.125" port: 34183 } } peers { permanent_uuid: "8ed6ff85c9f34eefa1e6e9539787f5c0" member_type: VOTER last_known_addr { host: "127.6.40.124" port: 39985 } }
I20260806 22:49:36.989207 25402 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ab2bc57725c4628adf19d0081ffe422 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6ab2bc57725c4628adf19d0081ffe422. Latest consensus state: current_term: 1 leader_uuid: "6ab2bc57725c4628adf19d0081ffe422" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c45a838d17b542ac9697f282b50e44e5" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 33885 } } peers { permanent_uuid: "6ab2bc57725c4628adf19d0081ffe422" member_type: VOTER last_known_addr { host: "127.6.40.125" port: 34183 } } peers { permanent_uuid: "8ed6ff85c9f34eefa1e6e9539787f5c0" member_type: VOTER last_known_addr { host: "127.6.40.124" port: 39985 } } }
I20260806 22:49:36.989346 25402 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ab2bc57725c4628adf19d0081ffe422 [sys.catalog]: This master's current role is: LEADER
I20260806 22:49:36.989630 25404 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:49:36.989871 25404 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:49:36.991331 25076 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c45a838d17b542ac9697f282b50e44e5 [term 1 FOLLOWER]: Refusing update from remote peer 6ab2bc57725c4628adf19d0081ffe422: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:49:36.991328 25291 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8ed6ff85c9f34eefa1e6e9539787f5c0 [term 1 FOLLOWER]: Refusing update from remote peer 6ab2bc57725c4628adf19d0081ffe422: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:49:36.991750 25383 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6ab2bc57725c4628adf19d0081ffe422 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45a838d17b542ac9697f282b50e44e5" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 33885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:49:36.991843 25402 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6ab2bc57725c4628adf19d0081ffe422 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ed6ff85c9f34eefa1e6e9539787f5c0" member_type: VOTER last_known_addr { host: "127.6.40.124" port: 39985 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:49:36.993173 25339 sys_catalog.cc:455] T 00000000000000000000000000000000 P c45a838d17b542ac9697f282b50e44e5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6ab2bc57725c4628adf19d0081ffe422. Latest consensus state: current_term: 1 leader_uuid: "6ab2bc57725c4628adf19d0081ffe422" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c45a838d17b542ac9697f282b50e44e5" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 33885 } } peers { permanent_uuid: "6ab2bc57725c4628adf19d0081ffe422" member_type: VOTER last_known_addr { host: "127.6.40.125" port: 34183 } } peers { permanent_uuid: "8ed6ff85c9f34eefa1e6e9539787f5c0" member_type: VOTER last_known_addr { host: "127.6.40.124" port: 39985 } } }
I20260806 22:49:36.993160 25356 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ed6ff85c9f34eefa1e6e9539787f5c0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6ab2bc57725c4628adf19d0081ffe422. Latest consensus state: current_term: 1 leader_uuid: "6ab2bc57725c4628adf19d0081ffe422" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c45a838d17b542ac9697f282b50e44e5" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 33885 } } peers { permanent_uuid: "6ab2bc57725c4628adf19d0081ffe422" member_type: VOTER last_known_addr { host: "127.6.40.125" port: 34183 } } peers { permanent_uuid: "8ed6ff85c9f34eefa1e6e9539787f5c0" member_type: VOTER last_known_addr { host: "127.6.40.124" port: 39985 } } }
I20260806 22:49:36.993252 25356 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ed6ff85c9f34eefa1e6e9539787f5c0 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:49:36.993252 25339 sys_catalog.cc:458] T 00000000000000000000000000000000 P c45a838d17b542ac9697f282b50e44e5 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:49:36.994464 25402 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ab2bc57725c4628adf19d0081ffe422 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6ab2bc57725c4628adf19d0081ffe422" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c45a838d17b542ac9697f282b50e44e5" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 33885 } } peers { permanent_uuid: "6ab2bc57725c4628adf19d0081ffe422" member_type: VOTER last_known_addr { host: "127.6.40.125" port: 34183 } } peers { permanent_uuid: "8ed6ff85c9f34eefa1e6e9539787f5c0" member_type: VOTER last_known_addr { host: "127.6.40.124" port: 39985 } } }
I20260806 22:49:36.994482 25356 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ed6ff85c9f34eefa1e6e9539787f5c0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6ab2bc57725c4628adf19d0081ffe422" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c45a838d17b542ac9697f282b50e44e5" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 33885 } } peers { permanent_uuid: "6ab2bc57725c4628adf19d0081ffe422" member_type: VOTER last_known_addr { host: "127.6.40.125" port: 34183 } } peers { permanent_uuid: "8ed6ff85c9f34eefa1e6e9539787f5c0" member_type: VOTER last_known_addr { host: "127.6.40.124" port: 39985 } } }
I20260806 22:49:36.994556 25356 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ed6ff85c9f34eefa1e6e9539787f5c0 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:49:36.994556 25402 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ab2bc57725c4628adf19d0081ffe422 [sys.catalog]: This master's current role is: LEADER
I20260806 22:49:36.994765 25405 mvcc.cc:205] Tried to move back new op lower bound from 7315687739354116096 to 7315687739346587648. Current Snapshot: MvccSnapshot[applied={T|T < 7315687739354116096}]
I20260806 22:49:36.995051 25383 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ab2bc57725c4628adf19d0081ffe422 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6ab2bc57725c4628adf19d0081ffe422" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c45a838d17b542ac9697f282b50e44e5" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 33885 } } peers { permanent_uuid: "6ab2bc57725c4628adf19d0081ffe422" member_type: VOTER last_known_addr { host: "127.6.40.125" port: 34183 } } peers { permanent_uuid: "8ed6ff85c9f34eefa1e6e9539787f5c0" member_type: VOTER last_known_addr { host: "127.6.40.124" port: 39985 } } }
I20260806 22:49:36.995131 25383 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ab2bc57725c4628adf19d0081ffe422 [sys.catalog]: This master's current role is: LEADER
I20260806 22:49:36.995180 25408 mvcc.cc:205] Tried to move back new op lower bound from 7315687739354116096 to 7315687739346587648. Current Snapshot: MvccSnapshot[applied={T|T < 7315687739354116096}]
I20260806 22:49:36.995175 25339 sys_catalog.cc:455] T 00000000000000000000000000000000 P c45a838d17b542ac9697f282b50e44e5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6ab2bc57725c4628adf19d0081ffe422" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c45a838d17b542ac9697f282b50e44e5" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 33885 } } peers { permanent_uuid: "6ab2bc57725c4628adf19d0081ffe422" member_type: VOTER last_known_addr { host: "127.6.40.125" port: 34183 } } peers { permanent_uuid: "8ed6ff85c9f34eefa1e6e9539787f5c0" member_type: VOTER last_known_addr { host: "127.6.40.124" port: 39985 } } }
I20260806 22:49:36.995271 25339 sys_catalog.cc:458] T 00000000000000000000000000000000 P c45a838d17b542ac9697f282b50e44e5 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:49:36.995860 25404 catalog_manager.cc:1347] Generated new cluster ID: 8f05d032c4c34d71b1b4ccde23b1bde3
I20260806 22:49:36.995920 25404 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:49:36.995704 25409 mvcc.cc:205] Tried to move back new op lower bound from 7315687739354116096 to 7315687739346587648. Current Snapshot: MvccSnapshot[applied={T|T < 7315687739354116096}]
I20260806 22:49:37.002252 25404 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:49:37.003088 25404 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:49:37.012666 25404 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6ab2bc57725c4628adf19d0081ffe422: Generated new TSK 0
I20260806 22:49:37.012971 25404 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:49:37.781385 25353 catalog_manager.cc:1259] Loaded cluster ID: 8f05d032c4c34d71b1b4ccde23b1bde3
I20260806 22:49:37.781472 25353 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c45a838d17b542ac9697f282b50e44e5: loading cluster ID for follower catalog manager: success
I20260806 22:49:37.782945 25353 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c45a838d17b542ac9697f282b50e44e5: acquiring CA information for follower catalog manager: success
I20260806 22:49:37.783360 25353 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c45a838d17b542ac9697f282b50e44e5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 22:49:37.798906 25370 catalog_manager.cc:1259] Loaded cluster ID: 8f05d032c4c34d71b1b4ccde23b1bde3
I20260806 22:49:37.798967 25370 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8ed6ff85c9f34eefa1e6e9539787f5c0: loading cluster ID for follower catalog manager: success
I20260806 22:49:37.799731 25370 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8ed6ff85c9f34eefa1e6e9539787f5c0: acquiring CA information for follower catalog manager: success
I20260806 22:49:37.799978 25370 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8ed6ff85c9f34eefa1e6e9539787f5c0: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 22:49:38.068359 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 25006
I20260806 22:49:38.081493 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 25119
I20260806 22:49:38.089228 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 25230
2026-08-06T22:49:38Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3170 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056405724609-6305-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:49:38 dist-test-slave-zh2d krb5kdc[25430](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:49:38 dist-test-slave-zh2d krb5kdc[25430](info): set up 2 sockets
Aug 06 22:49:38 dist-test-slave-zh2d krb5kdc[25430](info): commencing operation
krb5kdc: starting...
W20260806 22:49:39.311230 6305 mini_kdc.cc:121] Time spent starting KDC: real 1.198s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056405724609-6305-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056405724609-6305-0/krb5kdc/test-user.keytab.
Aug 06 22:49:39 dist-test-slave-zh2d krb5kdc[25430](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056579, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:49:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:49:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.40.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.40.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.40.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab.
Entry for principal kudu/127.6.40.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab.
I20260806 22:49:39.437783 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056405724609-6305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056405724609-6305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056405724609-6305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.126:35461
--webserver_interface=127.6.40.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab
--principal=kudu/127.6.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:37899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.40.126:35461
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:49:39.521548 25450 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:49:39.521756 25450 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:49:39.521778 25450 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:49:39.522828 25450 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:49:39.522862 25450 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:49:39.522877 25450 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:49:39.522889 25450 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:49:39.522902 25450 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:49:39.524276 25450 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:37899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056405724609-6305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056405724609-6305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.40.126:35461
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab
--principal=kudu/127.6.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.126:35461
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25450
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056405724609-6305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:49:39.524549 25450 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:49:39.524739 25450 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:49:39.529676 25459 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:39.529902 25450 server_base.cc:1034] running on GCE node
W20260806 22:49:39.529681 25463 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:39.529686 25460 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:39.530270 25450 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:49:39.530526 25450 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:49:39.531704 25450 hybrid_clock.cc:648] HybridClock initialized: now 1786056579531694 us; error 48 us; skew 500 ppm
Aug 06 22:49:39 dist-test-slave-zh2d krb5kdc[25430](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056579, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:49:39.539057 25450 init.cc:377] Logged in from keytab as kudu/127.6.40.126@KRBTEST.COM (short username kudu)
I20260806 22:49:39.539649 25450 webserver.cc:460] Webserver started at http://127.6.40.126:46277/ using document root <none> and password file <none>
I20260806 22:49:39.539947 25450 fs_manager.cc:359] Metadata directory not provided
I20260806 22:49:39.540028 25450 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:49:39.540202 25450 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:49:39.541525 25450 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056405724609-6305-0/minicluster-data/master-0/data/instance:
uuid: "c70aa8524d4f4c3dadebedc6dd4fcb67"
format_stamp: "Formatted at 2026-08-06 22:49:39 on dist-test-slave-zh2d"
server_key: "6cf4d56fbad54564e0e18b97a9f3efd3"
server_key_iv: "0a02a08f1864b430e55b6760e73f8948"
server_key_version: "encryptionkey@0"
I20260806 22:49:39.541970 25450 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056405724609-6305-0/minicluster-data/master-0/wal/instance:
uuid: "c70aa8524d4f4c3dadebedc6dd4fcb67"
format_stamp: "Formatted at 2026-08-06 22:49:39 on dist-test-slave-zh2d"
server_key: "6cf4d56fbad54564e0e18b97a9f3efd3"
server_key_iv: "0a02a08f1864b430e55b6760e73f8948"
server_key_version: "encryptionkey@0"
I20260806 22:49:39.543694 25450 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:49:39.544507 25475 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:39.544710 25450 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:49:39.544821 25450 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056405724609-6305-0/minicluster-data/master-0/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056405724609-6305-0/minicluster-data/master-0/wal
uuid: "c70aa8524d4f4c3dadebedc6dd4fcb67"
format_stamp: "Formatted at 2026-08-06 22:49:39 on dist-test-slave-zh2d"
server_key: "6cf4d56fbad54564e0e18b97a9f3efd3"
server_key_iv: "0a02a08f1864b430e55b6760e73f8948"
server_key_version: "encryptionkey@0"
I20260806 22:49:39.544955 25450 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056405724609-6305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056405724609-6305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056405724609-6305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:49:39.556636 25450 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:49:39.558702 25450 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:49:39.558879 25450 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:49:39.563434 25450 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.126:35461
I20260806 22:49:39.563489 25542 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.126:35461 every 8 connection(s)
I20260806 22:49:39.563853 25450 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
I20260806 22:49:39.564718 25543 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:49:39.567050 25543 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c70aa8524d4f4c3dadebedc6dd4fcb67: Bootstrap starting.
I20260806 22:49:39.567654 25543 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c70aa8524d4f4c3dadebedc6dd4fcb67: Neither blocks nor log segments found. Creating new log.
I20260806 22:49:39.567986 25543 log.cc:824] T 00000000000000000000000000000000 P c70aa8524d4f4c3dadebedc6dd4fcb67: Log is configured to *not* fsync() on all Append() calls
I20260806 22:49:39.568707 25543 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c70aa8524d4f4c3dadebedc6dd4fcb67: No bootstrap required, opened a new log
I20260806 22:49:39.570360 25543 raft_consensus.cc:348] T 00000000000000000000000000000000 P c70aa8524d4f4c3dadebedc6dd4fcb67 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c70aa8524d4f4c3dadebedc6dd4fcb67" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 35461 } }
I20260806 22:49:39.570498 25543 raft_consensus.cc:374] T 00000000000000000000000000000000 P c70aa8524d4f4c3dadebedc6dd4fcb67 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:49:39.570521 25543 raft_consensus.cc:729] T 00000000000000000000000000000000 P c70aa8524d4f4c3dadebedc6dd4fcb67 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c70aa8524d4f4c3dadebedc6dd4fcb67, State: Initialized, Role: FOLLOWER
I20260806 22:49:39.570686 25543 consensus_queue.cc:260] T 00000000000000000000000000000000 P c70aa8524d4f4c3dadebedc6dd4fcb67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c70aa8524d4f4c3dadebedc6dd4fcb67" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 35461 } }
I20260806 22:49:39.570753 25543 raft_consensus.cc:388] T 00000000000000000000000000000000 P c70aa8524d4f4c3dadebedc6dd4fcb67 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:49:39.570797 25543 raft_consensus.cc:482] T 00000000000000000000000000000000 P c70aa8524d4f4c3dadebedc6dd4fcb67 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:49:39.570848 25543 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c70aa8524d4f4c3dadebedc6dd4fcb67 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:49:39.571403 25543 raft_consensus.cc:504] T 00000000000000000000000000000000 P c70aa8524d4f4c3dadebedc6dd4fcb67 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c70aa8524d4f4c3dadebedc6dd4fcb67" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 35461 } }
I20260806 22:49:39.571518 25543 leader_election.cc:302] T 00000000000000000000000000000000 P c70aa8524d4f4c3dadebedc6dd4fcb67 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c70aa8524d4f4c3dadebedc6dd4fcb67; no voters:
I20260806 22:49:39.571717 25543 leader_election.cc:288] T 00000000000000000000000000000000 P c70aa8524d4f4c3dadebedc6dd4fcb67 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:49:39.571759 25547 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c70aa8524d4f4c3dadebedc6dd4fcb67 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:49:39.571942 25547 raft_consensus.cc:686] T 00000000000000000000000000000000 P c70aa8524d4f4c3dadebedc6dd4fcb67 [term 1 LEADER]: Becoming Leader. State: Replica: c70aa8524d4f4c3dadebedc6dd4fcb67, State: Running, Role: LEADER
I20260806 22:49:39.572001 25543 sys_catalog.cc:564] T 00000000000000000000000000000000 P c70aa8524d4f4c3dadebedc6dd4fcb67 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:49:39.572154 25547 consensus_queue.cc:237] T 00000000000000000000000000000000 P c70aa8524d4f4c3dadebedc6dd4fcb67 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c70aa8524d4f4c3dadebedc6dd4fcb67" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 35461 } }
I20260806 22:49:39.572589 25552 sys_catalog.cc:455] T 00000000000000000000000000000000 P c70aa8524d4f4c3dadebedc6dd4fcb67 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c70aa8524d4f4c3dadebedc6dd4fcb67. Latest consensus state: current_term: 1 leader_uuid: "c70aa8524d4f4c3dadebedc6dd4fcb67" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c70aa8524d4f4c3dadebedc6dd4fcb67" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 35461 } } }
I20260806 22:49:39.572722 25551 sys_catalog.cc:455] T 00000000000000000000000000000000 P c70aa8524d4f4c3dadebedc6dd4fcb67 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c70aa8524d4f4c3dadebedc6dd4fcb67" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c70aa8524d4f4c3dadebedc6dd4fcb67" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 35461 } } }
I20260806 22:49:39.572739 25552 sys_catalog.cc:458] T 00000000000000000000000000000000 P c70aa8524d4f4c3dadebedc6dd4fcb67 [sys.catalog]: This master's current role is: LEADER
I20260806 22:49:39.572818 25551 sys_catalog.cc:458] T 00000000000000000000000000000000 P c70aa8524d4f4c3dadebedc6dd4fcb67 [sys.catalog]: This master's current role is: LEADER
I20260806 22:49:39.573335 25563 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:49:39.573485 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 25450
I20260806 22:49:39.573611 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056405724609-6305-0/minicluster-data/master-0/wal/instance
I20260806 22:49:39.573850 6305 external_mini_cluster.cc:1428] Setting key 46deff4590ff6f4ecacba1bd83d9c5f9
I20260806 22:49:39.573889 25563 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:49:39.575486 25563 catalog_manager.cc:1347] Generated new cluster ID: f897f032dffc4e45b93bdff75e698114
I20260806 22:49:39.575543 25563 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 22:49:39 dist-test-slave-zh2d krb5kdc[25430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056579, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
I20260806 22:49:39.585481 25579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:39.574326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44856 (local address 127.6.40.126:35461)
0806 22:49:39.574519 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:39.574529 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:39.574614 (+ 85us) server_negotiation.cc:407] Connection header received
0806 22:49:39.574785 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:39.574790 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:39.574837 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:39.575000 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:39.577293 (+ 2293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:39.578211 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:39.579019 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:39.579434 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:39.579562 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:39.581976 (+ 2414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:39.581980 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:39.581980 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:39.581993 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:39.583701 (+ 1708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:39.584077 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:39.584080 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:39.584081 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:39.584194 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:39.584557 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:39.584558 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:39.584559 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:39.584809 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:39.585037 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:39.585107 (+ 70us) server_negotiation.cc:299] Negotiation successful
0806 22:49:39.585252 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":89,"threads_started":1}
I20260806 22:49:39.586592 6305 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:49:39.586655 25563 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:49:39.587342 25563 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:49:39.594924 25563 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c70aa8524d4f4c3dadebedc6dd4fcb67: Generated new TSK 0
I20260806 22:49:39.595103 25563 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:49:39.640663 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 25450
2026-08-06T22:49:39Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1551 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:49:39 dist-test-slave-zh2d krb5kdc[25586](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:49:39 dist-test-slave-zh2d krb5kdc[25586](info): set up 2 sockets
Aug 06 22:49:39 dist-test-slave-zh2d krb5kdc[25586](info): commencing operation
krb5kdc: starting...
W20260806 22:49:40.858776 6305 mini_kdc.cc:121] Time spent starting KDC: real 1.194s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/krb5kdc/test-user.keytab.
Aug 06 22:49:40 dist-test-slave-zh2d krb5kdc[25586](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056580, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:49:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:49:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.40.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.40.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.40.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab.
Entry for principal kudu/127.6.40.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab.
I20260806 22:49:40.987285 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.126:32939
--webserver_interface=127.6.40.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab
--principal=kudu/127.6.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:42369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.40.126:32939
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:49:41.070688 25604 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:49:41.070897 25604 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:49:41.070920 25604 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:49:41.071985 25604 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:49:41.072021 25604 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:49:41.072042 25604 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:49:41.072054 25604 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:49:41.072067 25604 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:49:41.073509 25604 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:42369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.40.126:32939
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab
--principal=kudu/127.6.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.126:32939
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25604
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:49:41.073712 25604 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:49:41.073879 25604 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:49:41.078800 25611 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:41.078779 25615 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:41.078778 25610 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:41.079236 25604 server_base.cc:1034] running on GCE node
I20260806 22:49:41.079531 25604 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:49:41.079825 25604 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:49:41.080991 25604 hybrid_clock.cc:648] HybridClock initialized: now 1786056581080977 us; error 45 us; skew 500 ppm
Aug 06 22:49:41 dist-test-slave-zh2d krb5kdc[25586](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056581, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:49:41.083905 25604 init.cc:377] Logged in from keytab as kudu/127.6.40.126@KRBTEST.COM (short username kudu)
I20260806 22:49:41.084345 25604 webserver.cc:460] Webserver started at http://127.6.40.126:45679/ using document root <none> and password file <none>
I20260806 22:49:41.084579 25604 fs_manager.cc:359] Metadata directory not provided
I20260806 22:49:41.084648 25604 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:49:41.084815 25604 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:49:41.085903 25604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/master-0/data/instance:
uuid: "b9e1eb63b32f4e9ea1c7f100f2d38d4c"
format_stamp: "Formatted at 2026-08-06 22:49:41 on dist-test-slave-zh2d"
server_key: "96801279d1363050246285ee624aa72c"
server_key_iv: "501ed5420d49cdc58d4f18edeeeed1a7"
server_key_version: "encryptionkey@0"
I20260806 22:49:41.086275 25604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/master-0/wal/instance:
uuid: "b9e1eb63b32f4e9ea1c7f100f2d38d4c"
format_stamp: "Formatted at 2026-08-06 22:49:41 on dist-test-slave-zh2d"
server_key: "96801279d1363050246285ee624aa72c"
server_key_iv: "501ed5420d49cdc58d4f18edeeeed1a7"
server_key_version: "encryptionkey@0"
I20260806 22:49:41.087643 25604 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 22:49:41.088581 25626 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:41.088769 25604 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:49:41.088891 25604 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/master-0/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/master-0/wal
uuid: "b9e1eb63b32f4e9ea1c7f100f2d38d4c"
format_stamp: "Formatted at 2026-08-06 22:49:41 on dist-test-slave-zh2d"
server_key: "96801279d1363050246285ee624aa72c"
server_key_iv: "501ed5420d49cdc58d4f18edeeeed1a7"
server_key_version: "encryptionkey@0"
I20260806 22:49:41.088984 25604 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:49:41.106835 25604 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:49:41.109277 25604 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:49:41.109452 25604 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:49:41.113656 25604 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.126:32939
I20260806 22:49:41.113698 25702 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.126:32939 every 8 connection(s)
I20260806 22:49:41.114086 25604 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
I20260806 22:49:41.115128 25703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:49:41.117509 25703 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9e1eb63b32f4e9ea1c7f100f2d38d4c: Bootstrap starting.
I20260806 22:49:41.118068 25703 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b9e1eb63b32f4e9ea1c7f100f2d38d4c: Neither blocks nor log segments found. Creating new log.
I20260806 22:49:41.118322 25703 log.cc:824] T 00000000000000000000000000000000 P b9e1eb63b32f4e9ea1c7f100f2d38d4c: Log is configured to *not* fsync() on all Append() calls
I20260806 22:49:41.119033 25703 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9e1eb63b32f4e9ea1c7f100f2d38d4c: No bootstrap required, opened a new log
I20260806 22:49:41.120605 25703 raft_consensus.cc:348] T 00000000000000000000000000000000 P b9e1eb63b32f4e9ea1c7f100f2d38d4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9e1eb63b32f4e9ea1c7f100f2d38d4c" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 32939 } }
I20260806 22:49:41.120743 25703 raft_consensus.cc:374] T 00000000000000000000000000000000 P b9e1eb63b32f4e9ea1c7f100f2d38d4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:49:41.120766 25703 raft_consensus.cc:729] T 00000000000000000000000000000000 P b9e1eb63b32f4e9ea1c7f100f2d38d4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9e1eb63b32f4e9ea1c7f100f2d38d4c, State: Initialized, Role: FOLLOWER
I20260806 22:49:41.120930 25703 consensus_queue.cc:260] T 00000000000000000000000000000000 P b9e1eb63b32f4e9ea1c7f100f2d38d4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9e1eb63b32f4e9ea1c7f100f2d38d4c" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 32939 } }
I20260806 22:49:41.121016 25703 raft_consensus.cc:388] T 00000000000000000000000000000000 P b9e1eb63b32f4e9ea1c7f100f2d38d4c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:49:41.121045 25703 raft_consensus.cc:482] T 00000000000000000000000000000000 P b9e1eb63b32f4e9ea1c7f100f2d38d4c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:49:41.121083 25703 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b9e1eb63b32f4e9ea1c7f100f2d38d4c [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:49:41.121862 25703 raft_consensus.cc:504] T 00000000000000000000000000000000 P b9e1eb63b32f4e9ea1c7f100f2d38d4c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9e1eb63b32f4e9ea1c7f100f2d38d4c" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 32939 } }
I20260806 22:49:41.121953 25703 leader_election.cc:302] T 00000000000000000000000000000000 P b9e1eb63b32f4e9ea1c7f100f2d38d4c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b9e1eb63b32f4e9ea1c7f100f2d38d4c; no voters:
I20260806 22:49:41.122077 25703 leader_election.cc:288] T 00000000000000000000000000000000 P b9e1eb63b32f4e9ea1c7f100f2d38d4c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:49:41.122159 25706 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b9e1eb63b32f4e9ea1c7f100f2d38d4c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:49:41.122378 25706 raft_consensus.cc:686] T 00000000000000000000000000000000 P b9e1eb63b32f4e9ea1c7f100f2d38d4c [term 1 LEADER]: Becoming Leader. State: Replica: b9e1eb63b32f4e9ea1c7f100f2d38d4c, State: Running, Role: LEADER
I20260806 22:49:41.122386 25703 sys_catalog.cc:564] T 00000000000000000000000000000000 P b9e1eb63b32f4e9ea1c7f100f2d38d4c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:49:41.122766 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 25604
I20260806 22:49:41.122892 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/master-0/wal/instance
I20260806 22:49:41.123082 6305 external_mini_cluster.cc:1428] Setting key bcaa3853fb1c1a7a0e48afc448608d06
I20260806 22:49:41.123202 25706 consensus_queue.cc:237] T 00000000000000000000000000000000 P b9e1eb63b32f4e9ea1c7f100f2d38d4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9e1eb63b32f4e9ea1c7f100f2d38d4c" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 32939 } }
I20260806 22:49:41.123898 25707 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9e1eb63b32f4e9ea1c7f100f2d38d4c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b9e1eb63b32f4e9ea1c7f100f2d38d4c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9e1eb63b32f4e9ea1c7f100f2d38d4c" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 32939 } } }
I20260806 22:49:41.124186 25707 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9e1eb63b32f4e9ea1c7f100f2d38d4c [sys.catalog]: This master's current role is: LEADER
I20260806 22:49:41.124053 25708 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9e1eb63b32f4e9ea1c7f100f2d38d4c [sys.catalog]: SysCatalogTable state changed. Reason: New leader b9e1eb63b32f4e9ea1c7f100f2d38d4c. Latest consensus state: current_term: 1 leader_uuid: "b9e1eb63b32f4e9ea1c7f100f2d38d4c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9e1eb63b32f4e9ea1c7f100f2d38d4c" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 32939 } } }
I20260806 22:49:41.124254 25708 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9e1eb63b32f4e9ea1c7f100f2d38d4c [sys.catalog]: This master's current role is: LEADER
W20260806 22:49:41.124936 25724 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b9e1eb63b32f4e9ea1c7f100f2d38d4c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:49:41.124997 25724 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:49:41.125053 25726 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:49:41.125217 25726 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:49:41.126937 25726 catalog_manager.cc:1347] Generated new cluster ID: bf2abec72c424d718a8be5897a9eb258
I20260806 22:49:41.126993 25726 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 22:49:41 dist-test-slave-zh2d krb5kdc[25586](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056580, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
I20260806 22:49:41.135628 25725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:41.123838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51224 (local address 127.6.40.126:32939)
0806 22:49:41.124129 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:41.124138 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:41.124170 (+ 32us) server_negotiation.cc:407] Connection header received
0806 22:49:41.124297 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:41.124304 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:41.124379 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:41.124528 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:41.125356 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:41.126710 (+ 1354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:41.127793 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:41.128456 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:41.128583 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:41.131481 (+ 2898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:41.131485 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:41.131485 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:41.131498 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:41.133953 (+ 2455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:41.134370 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:41.134373 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:41.134373 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:41.134490 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:41.134708 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:41.134708 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:41.134709 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:41.135004 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:41.135151 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:41.135277 (+ 126us) server_negotiation.cc:299] Negotiation successful
0806 22:49:41.135411 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":243,"thread_start_us":90,"threads_started":1}
I20260806 22:49:41.164971 25726 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:49:41.165617 25726 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:49:41.171084 25726 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b9e1eb63b32f4e9ea1c7f100f2d38d4c: Generated new TSK 0
I20260806 22:49:41.171264 25726 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.40.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.40.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.40.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.65.keytab.
Entry for principal kudu/127.6.40.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.65.keytab.
I20260806 22:49:41.211259 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.65:0
--local_ip_for_outbound_sockets=127.6.40.65
--webserver_interface=127.6.40.65
--webserver_port=0
--tserver_master_addrs=127.6.40.126:32939
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.65.keytab
--principal=kudu/127.6.40.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:42369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:49:41.321154 25734 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:49:41.321358 25734 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:49:41.321382 25734 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:49:41.322511 25734 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:49:41.322551 25734 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:49:41.322577 25734 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.65
I20260806 22:49:41.324134 25734 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:42369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.65.keytab
--principal=kudu/127.6.40.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.40.65
--webserver_port=0
--tserver_master_addrs=127.6.40.126:32939
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25734
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.65
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:49:41.324347 25734 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:49:41.324535 25734 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:49:41.329665 25744 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:41.329675 25747 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:41.329749 25734 server_base.cc:1034] running on GCE node
W20260806 22:49:41.329675 25741 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:41.330214 25734 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:49:41.330510 25734 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:49:41.331656 25734 hybrid_clock.cc:648] HybridClock initialized: now 1786056581331644 us; error 30 us; skew 500 ppm
Aug 06 22:49:41 dist-test-slave-zh2d krb5kdc[25586](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056581, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:49:41.334298 25734 init.cc:377] Logged in from keytab as kudu/127.6.40.65@KRBTEST.COM (short username kudu)
I20260806 22:49:41.334721 25734 webserver.cc:460] Webserver started at http://127.6.40.65:38705/ using document root <none> and password file <none>
I20260806 22:49:41.334962 25734 fs_manager.cc:359] Metadata directory not provided
I20260806 22:49:41.335008 25734 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:49:41.335186 25734 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:49:41.336200 25734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-0/data/instance:
uuid: "fd1b2e1656e24421813074d3a7a3f6bb"
format_stamp: "Formatted at 2026-08-06 22:49:41 on dist-test-slave-zh2d"
server_key: "1debb8747148c3197bcef363e7b0d86f"
server_key_iv: "5852b2f46bb327f4ea1ba328b32abaad"
server_key_version: "encryptionkey@0"
I20260806 22:49:41.336558 25734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-0/wal/instance:
uuid: "fd1b2e1656e24421813074d3a7a3f6bb"
format_stamp: "Formatted at 2026-08-06 22:49:41 on dist-test-slave-zh2d"
server_key: "1debb8747148c3197bcef363e7b0d86f"
server_key_iv: "5852b2f46bb327f4ea1ba328b32abaad"
server_key_version: "encryptionkey@0"
I20260806 22:49:41.338083 25734 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:49:41.338814 25757 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:41.339013 25734 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:49:41.339123 25734 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-0/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-0/wal
uuid: "fd1b2e1656e24421813074d3a7a3f6bb"
format_stamp: "Formatted at 2026-08-06 22:49:41 on dist-test-slave-zh2d"
server_key: "1debb8747148c3197bcef363e7b0d86f"
server_key_iv: "5852b2f46bb327f4ea1ba328b32abaad"
server_key_version: "encryptionkey@0"
I20260806 22:49:41.339200 25734 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:49:41.355724 25734 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:49:41.357743 25734 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:49:41.357928 25734 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:49:41.358178 25734 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:49:41.358489 25734 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:49:41.358521 25734 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:41.358568 25734 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:49:41.358596 25734 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:41.364612 25734 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.65:43385
I20260806 22:49:41.364714 25922 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.65:43385 every 8 connection(s)
I20260806 22:49:41.365052 25734 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
I20260806 22:49:41.367995 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 25734
I20260806 22:49:41.368132 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-0/wal/instance
I20260806 22:49:41.368353 6305 external_mini_cluster.cc:1428] Setting key 37c1925e5b62e93351e4d949cd9af245
Aug 06 22:49:41 dist-test-slave-zh2d krb5kdc[25586](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056581, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.65@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
I20260806 22:49:41.375550 25725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:41.365797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.65:60939 (local address 127.6.40.126:32939)
0806 22:49:41.365846 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:41.365848 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:41.366057 (+ 209us) server_negotiation.cc:407] Connection header received
0806 22:49:41.366567 (+ 510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:41.366568 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:41.366579 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:41.366638 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:41.367567 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:41.368179 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:41.369589 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:41.369778 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:41.369834 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:41.372607 (+ 2773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:41.372608 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:41.372609 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:41.372616 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:41.374110 (+ 1494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:41.374548 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:41.374549 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:41.374549 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:41.374611 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:41.374878 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:41.374879 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:41.374879 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:41.375002 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:41.375074 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:41.375359 (+ 285us) server_negotiation.cc:299] Negotiation successful
0806 22:49:41.375454 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 22:49:41.375759 25928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:41.365753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:32939 (local address 127.6.40.65:60939)
0806 22:49:41.365921 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:41.365965 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:41.366477 (+ 512us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:41.366704 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:41.366707 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:41.367291 (+ 584us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:49:41.367479 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:41.367483 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:41.368270 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:41.368272 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:41.369503 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:41.369506 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:41.369868 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:41.369869 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:41.370023 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:41.370527 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:49:41.370539 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:49:41.372511 (+ 1972us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:49:41.374213 (+ 1702us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:41.374221 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:41.374223 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:41.374467 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:41.374686 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:41.374687 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:41.374687 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:41.374803 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:41.375078 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:49:41.375086 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:49:41.375226 (+ 140us) client_negotiation.cc:769] Sending connection context
0806 22:49:41.375335 (+ 109us) client_negotiation.cc:240] Negotiation successful
0806 22:49:41.375511 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":94,"threads_started":1}
I20260806 22:49:41.376449 25924 heartbeater.cc:347] Connected to a master server at 127.6.40.126:32939
I20260806 22:49:41.376533 25924 heartbeater.cc:464] Registering TS with master...
I20260806 22:49:41.376725 25924 heartbeater.cc:511] Master 127.6.40.126:32939 requested a full tablet report, sending...
I20260806 22:49:41.377310 25652 ts_manager.cc:194] Registered new tserver with Master: fd1b2e1656e24421813074d3a7a3f6bb (127.6.40.65:43385)
WARNING: no policy specified for kudu/127.6.40.66@KRBTEST.COM; defaulting to no policy
I20260806 22:49:41.377987 25652 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.40.65@KRBTEST.COM'} at 127.6.40.65:60939
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.40.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.40.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.66.keytab.
Entry for principal kudu/127.6.40.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.66.keytab.
I20260806 22:49:41.396241 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.66:0
--local_ip_for_outbound_sockets=127.6.40.66
--webserver_interface=127.6.40.66
--webserver_port=0
--tserver_master_addrs=127.6.40.126:32939
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.66.keytab
--principal=kudu/127.6.40.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:42369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:49:41.480479 25937 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:49:41.480677 25937 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:49:41.480700 25937 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:49:41.481897 25937 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:49:41.481947 25937 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:49:41.481973 25937 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.66
I20260806 22:49:41.483368 25937 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:42369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.66.keytab
--principal=kudu/127.6.40.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.40.66
--webserver_port=0
--tserver_master_addrs=127.6.40.126:32939
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25937
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.66
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:49:41.483557 25937 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:49:41.483736 25937 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:49:41.489065 25947 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:41.489202 25946 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:41.489104 25949 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:41.489089 25937 server_base.cc:1034] running on GCE node
I20260806 22:49:41.489605 25937 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:49:41.489825 25937 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:49:41.490993 25937 hybrid_clock.cc:648] HybridClock initialized: now 1786056581490985 us; error 36 us; skew 500 ppm
Aug 06 22:49:41 dist-test-slave-zh2d krb5kdc[25586](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056581, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:49:41.493666 25937 init.cc:377] Logged in from keytab as kudu/127.6.40.66@KRBTEST.COM (short username kudu)
I20260806 22:49:41.494135 25937 webserver.cc:460] Webserver started at http://127.6.40.66:44203/ using document root <none> and password file <none>
I20260806 22:49:41.494334 25937 fs_manager.cc:359] Metadata directory not provided
I20260806 22:49:41.494375 25937 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:49:41.494467 25937 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:49:41.495488 25937 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-1/data/instance:
uuid: "91e4eeee79304b0da16d36864b9fcc1f"
format_stamp: "Formatted at 2026-08-06 22:49:41 on dist-test-slave-zh2d"
server_key: "9319cfeecf264e9c4274d57891c2e5a9"
server_key_iv: "23407928f59f72f10f6975735c620a85"
server_key_version: "encryptionkey@0"
I20260806 22:49:41.495787 25937 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-1/wal/instance:
uuid: "91e4eeee79304b0da16d36864b9fcc1f"
format_stamp: "Formatted at 2026-08-06 22:49:41 on dist-test-slave-zh2d"
server_key: "9319cfeecf264e9c4274d57891c2e5a9"
server_key_iv: "23407928f59f72f10f6975735c620a85"
server_key_version: "encryptionkey@0"
I20260806 22:49:41.497175 25937 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:49:41.497880 25956 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:41.498061 25937 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:49:41.498159 25937 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-1/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-1/wal
uuid: "91e4eeee79304b0da16d36864b9fcc1f"
format_stamp: "Formatted at 2026-08-06 22:49:41 on dist-test-slave-zh2d"
server_key: "9319cfeecf264e9c4274d57891c2e5a9"
server_key_iv: "23407928f59f72f10f6975735c620a85"
server_key_version: "encryptionkey@0"
I20260806 22:49:41.498220 25937 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:49:41.513458 25937 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:49:41.515425 25937 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:49:41.515549 25937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:49:41.515751 25937 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:49:41.516077 25937 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:49:41.516117 25937 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:41.516140 25937 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:49:41.516187 25937 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:41.522533 25937 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.66:38635
I20260806 22:49:41.522608 26120 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.66:38635 every 8 connection(s)
I20260806 22:49:41.522998 25937 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
Aug 06 22:49:41 dist-test-slave-zh2d krb5kdc[25586](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056581, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.66@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
I20260806 22:49:41.531764 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 25937
I20260806 22:49:41.531885 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-1/wal/instance
I20260806 22:49:41.532122 6305 external_mini_cluster.cc:1428] Setting key b933e5c4e50c64b6685eff52bbe8cf83
I20260806 22:49:41.532625 25725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:41.523779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.66:59445 (local address 127.6.40.126:32939)
0806 22:49:41.523832 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:41.523834 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:41.524059 (+ 225us) server_negotiation.cc:407] Connection header received
0806 22:49:41.524617 (+ 558us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:41.524618 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:41.524631 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:41.524685 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:41.525682 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:41.526189 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:41.527233 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:41.527454 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:41.527531 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:41.529767 (+ 2236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:41.529768 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:41.529769 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:41.529777 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:41.531187 (+ 1410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:41.531566 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:41.531567 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:41.531567 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:41.531638 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:41.531891 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:41.531893 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:41.531893 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:41.532043 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:41.532119 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:41.532458 (+ 339us) server_negotiation.cc:299] Negotiation successful
0806 22:49:41.532562 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 22:49:41.532831 26125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:41.523744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:32939 (local address 127.6.40.66:59445)
0806 22:49:41.523897 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:41.523975 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:41.524505 (+ 530us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:41.524756 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:41.524760 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:41.525368 (+ 608us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:49:41.525575 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:41.525580 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:41.526287 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:41.526289 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:41.527137 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:41.527139 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:41.527541 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:41.527542 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:41.527682 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:41.528152 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:49:41.528160 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:49:41.529672 (+ 1512us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:49:41.531262 (+ 1590us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:41.531268 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:41.531270 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:41.531476 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:41.531707 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:41.531708 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:41.531708 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:41.531802 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:41.532115 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:49:41.532120 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:49:41.532273 (+ 153us) client_negotiation.cc:769] Sending connection context
0806 22:49:41.532404 (+ 131us) client_negotiation.cc:240] Negotiation successful
0806 22:49:41.532604 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":72,"threads_started":1}
I20260806 22:49:41.533383 26123 heartbeater.cc:347] Connected to a master server at 127.6.40.126:32939
I20260806 22:49:41.533488 26123 heartbeater.cc:464] Registering TS with master...
I20260806 22:49:41.533694 26123 heartbeater.cc:511] Master 127.6.40.126:32939 requested a full tablet report, sending...
I20260806 22:49:41.534082 25652 ts_manager.cc:194] Registered new tserver with Master: 91e4eeee79304b0da16d36864b9fcc1f (127.6.40.66:38635)
I20260806 22:49:41.534600 25652 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.40.66@KRBTEST.COM'} at 127.6.40.66:59445
WARNING: no policy specified for kudu/127.6.40.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.40.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.40.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.67.keytab.
Entry for principal kudu/127.6.40.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.67.keytab.
I20260806 22:49:41.559388 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.67:0
--local_ip_for_outbound_sockets=127.6.40.67
--webserver_interface=127.6.40.67
--webserver_port=0
--tserver_master_addrs=127.6.40.126:32939
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.67.keytab
--principal=kudu/127.6.40.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:42369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:49:41.642958 26129 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:49:41.643184 26129 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:49:41.643225 26129 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:49:41.644380 26129 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:49:41.644452 26129 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:49:41.644498 26129 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.67
I20260806 22:49:41.646003 26129 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:42369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.67.keytab
--principal=kudu/127.6.40.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.40.67
--webserver_port=0
--tserver_master_addrs=127.6.40.126:32939
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26129
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.67
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:49:41.646271 26129 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:49:41.646507 26129 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:49:41.651700 26137 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:41.651723 26136 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:41.651799 26129 server_base.cc:1034] running on GCE node
W20260806 22:49:41.651729 26139 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:41.652098 26129 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:49:41.652495 26129 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:49:41.653679 26129 hybrid_clock.cc:648] HybridClock initialized: now 1786056581653656 us; error 51 us; skew 500 ppm
Aug 06 22:49:41 dist-test-slave-zh2d krb5kdc[25586](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056581, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:49:41.656529 26129 init.cc:377] Logged in from keytab as kudu/127.6.40.67@KRBTEST.COM (short username kudu)
I20260806 22:49:41.657119 26129 webserver.cc:460] Webserver started at http://127.6.40.67:37237/ using document root <none> and password file <none>
I20260806 22:49:41.657390 26129 fs_manager.cc:359] Metadata directory not provided
I20260806 22:49:41.657471 26129 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:49:41.657653 26129 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:49:41.658847 26129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-2/data/instance:
uuid: "87fd367889cc4d379ad9e432f8572051"
format_stamp: "Formatted at 2026-08-06 22:49:41 on dist-test-slave-zh2d"
server_key: "cb95ff67629c4c6a77b7f7ae9809561c"
server_key_iv: "5dc2f637fd29ccca8c0c4edb225f396b"
server_key_version: "encryptionkey@0"
I20260806 22:49:41.659320 26129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-2/wal/instance:
uuid: "87fd367889cc4d379ad9e432f8572051"
format_stamp: "Formatted at 2026-08-06 22:49:41 on dist-test-slave-zh2d"
server_key: "cb95ff67629c4c6a77b7f7ae9809561c"
server_key_iv: "5dc2f637fd29ccca8c0c4edb225f396b"
server_key_version: "encryptionkey@0"
I20260806 22:49:41.660959 26129 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260806 22:49:41.661827 26147 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:41.662020 26129 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:49:41.662147 26129 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-2/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-2/wal
uuid: "87fd367889cc4d379ad9e432f8572051"
format_stamp: "Formatted at 2026-08-06 22:49:41 on dist-test-slave-zh2d"
server_key: "cb95ff67629c4c6a77b7f7ae9809561c"
server_key_iv: "5dc2f637fd29ccca8c0c4edb225f396b"
server_key_version: "encryptionkey@0"
I20260806 22:49:41.662235 26129 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:49:41.684155 26129 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:49:41.686899 26129 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:49:41.687088 26129 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:49:41.687362 26129 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:49:41.687698 26129 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:49:41.687753 26129 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:41.687800 26129 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:49:41.687824 26129 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:41.694610 26129 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.67:46713
I20260806 22:49:41.694700 26316 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.67:46713 every 8 connection(s)
I20260806 22:49:41.695055 26129 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
Aug 06 22:49:41 dist-test-slave-zh2d krb5kdc[25586](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056581, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.67@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
I20260806 22:49:41.704757 25725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:41.695883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.67:59095 (local address 127.6.40.126:32939)
0806 22:49:41.695957 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:41.695959 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:41.696166 (+ 207us) server_negotiation.cc:407] Connection header received
0806 22:49:41.696663 (+ 497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:41.696664 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:41.696675 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:41.696745 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:41.697761 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:41.698300 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:41.699301 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:41.699511 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:41.699575 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:41.701928 (+ 2353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:41.701930 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:41.701930 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:41.701938 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:41.703427 (+ 1489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:41.703847 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:41.703848 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:41.703848 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:41.703904 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:41.704133 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:41.704134 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:41.704134 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:41.704241 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:41.704293 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:41.704562 (+ 269us) server_negotiation.cc:299] Negotiation successful
0806 22:49:41.704631 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 22:49:41.704898 26320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:41.695863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:32939 (local address 127.6.40.67:59095)
0806 22:49:41.696032 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:41.696076 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:41.696577 (+ 501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:41.696800 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:41.696804 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:41.697461 (+ 657us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:49:41.697674 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:41.697679 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:41.698388 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:41.698388 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:41.699213 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:41.699215 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:41.699596 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:41.699597 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:41.699701 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:41.700264 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:49:41.700272 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:49:41.701823 (+ 1551us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:49:41.703527 (+ 1704us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:41.703534 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:41.703536 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:41.703774 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:41.703968 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:41.703969 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:41.703969 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:41.704077 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:41.704307 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:49:41.704312 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:49:41.704438 (+ 126us) client_negotiation.cc:769] Sending connection context
0806 22:49:41.704523 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 22:49:41.704650 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":87,"threads_started":1}
I20260806 22:49:41.704897 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 26129
I20260806 22:49:41.704972 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-2/wal/instance
I20260806 22:49:41.705137 6305 external_mini_cluster.cc:1428] Setting key e1bfd54d48b666405d9ddd84b2237c36
I20260806 22:49:41.705482 26317 heartbeater.cc:347] Connected to a master server at 127.6.40.126:32939
I20260806 22:49:41.705603 26317 heartbeater.cc:464] Registering TS with master...
I20260806 22:49:41.705786 26317 heartbeater.cc:511] Master 127.6.40.126:32939 requested a full tablet report, sending...
I20260806 22:49:41.706193 25652 ts_manager.cc:194] Registered new tserver with Master: 87fd367889cc4d379ad9e432f8572051 (127.6.40.67:46713)
I20260806 22:49:41.706537 25652 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.40.67@KRBTEST.COM'} at 127.6.40.67:59095
I20260806 22:49:41.716951 6305 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:49:41.724810 25725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:41.718365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51238 (local address 127.6.40.126:32939)
0806 22:49:41.718430 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:41.718431 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:41.718545 (+ 114us) server_negotiation.cc:407] Connection header received
0806 22:49:41.718583 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:41.718583 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:41.718592 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:41.718642 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:41.719196 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:41.719670 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:41.720461 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:41.720644 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:41.720690 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:41.722018 (+ 1328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:41.722020 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:41.722020 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:41.722027 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:41.723647 (+ 1620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:41.723986 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:41.723987 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:41.723988 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:41.724068 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:41.724249 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:41.724250 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:41.724250 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:41.724404 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:41.724511 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:41.724620 (+ 109us) server_negotiation.cc:299] Negotiation successful
0806 22:49:41.724687 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 22:49:41.726586 25652 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51238:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:49:41.728291 25652 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:49:41.735394 26334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:41.731121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.67:46713 (local address 127.0.0.1:39304)
0806 22:49:41.731374 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:41.731384 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:41.731425 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:41.732090 (+ 665us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:41.732091 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:41.732093 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:49:41.732142 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:41.732144 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:41.733192 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:41.733193 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:41.734305 (+ 1112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:41.734308 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:41.735101 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:41.735102 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:41.735163 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:41.735196 (+ 33us) client_negotiation.cc:769] Sending connection context
0806 22:49:41.735286 (+ 90us) client_negotiation.cc:240] Negotiation successful
0806 22:49:41.735333 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"mutex_wait_us":152,"thread_start_us":83,"threads_started":1}
I20260806 22:49:41.735435 26339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:41.731240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39304 (local address 127.6.40.67:46713)
0806 22:49:41.731932 (+ 692us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:41.731937 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:41.731946 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:49:41.731979 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:41.731980 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:41.731993 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:41.732092 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:49:41.732209 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:41.733088 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:41.734410 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:41.735013 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:41.735093 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:41.735155 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:41.735302 (+ 147us) server_negotiation.cc:299] Negotiation successful
0806 22:49:41.735348 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":656,"thread_start_us":103,"threads_started":1}
I20260806 22:49:41.736202 26220 tablet_service.cc:1467] Processing CreateTablet for tablet 73d44206237a460bb2e0b3e02554b4e1 (DEFAULT_TABLE table=test-table [id=9ba289e88a584d43bb71ad2b31e07007]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:49:41.736618 26220 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73d44206237a460bb2e0b3e02554b4e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:49:41.736828 26338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:41.730866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.66:38635 (local address 127.0.0.1:48992)
0806 22:49:41.731261 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:41.731274 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:41.731337 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:41.731733 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:41.731734 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:41.731737 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:49:41.731786 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:41.731787 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:41.733089 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:41.733091 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:41.734728 (+ 1637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:41.734732 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:41.736539 (+ 1807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:41.736540 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:41.736609 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:41.736613 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:49:41.736690 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 22:49:41.736735 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":373,"mutex_wait_us":141,"thread_start_us":102,"threads_started":1}
I20260806 22:49:41.737037 26337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:41.731004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36554 (local address 127.6.40.65:43385)
0806 22:49:41.731309 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:41.731313 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:41.731324 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:49:41.731490 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:41.731493 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:41.731508 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:41.731586 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:49:41.731764 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:41.732967 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:41.734827 (+ 1860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:41.735567 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:41.735642 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:41.735698 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:41.736898 (+ 1200us) server_negotiation.cc:299] Negotiation successful
0806 22:49:41.736942 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265,"thread_start_us":101,"threads_started":1}
I20260806 22:49:41.737108 26336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:41.731005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48992 (local address 127.6.40.66:38635)
0806 22:49:41.731218 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:41.731223 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:41.731469 (+ 246us) server_negotiation.cc:407] Connection header received
0806 22:49:41.731528 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:41.731531 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:41.731550 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:41.731639 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:49:41.731859 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:41.732971 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:41.735737 (+ 2766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:41.736451 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:41.736522 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:41.736577 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:41.736806 (+ 229us) server_negotiation.cc:299] Negotiation successful
0806 22:49:41.736899 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":91,"threads_started":1}
I20260806 22:49:41.737293 26333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:41.730866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.65:43385 (local address 127.0.0.1:36554)
0806 22:49:41.731212 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:41.731259 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:41.731335 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:41.731597 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:41.731600 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:41.731603 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:49:41.731681 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:41.731684 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:41.733303 (+ 1619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:41.733304 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:41.734729 (+ 1425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:41.734732 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:41.736620 (+ 1888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:41.736621 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:41.736690 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:41.736695 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:49:41.736751 (+ 56us) client_negotiation.cc:240] Negotiation successful
0806 22:49:41.736798 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"mutex_wait_us":23,"thread_start_us":67,"threads_started":1}
I20260806 22:49:41.737834 26024 tablet_service.cc:1467] Processing CreateTablet for tablet 73d44206237a460bb2e0b3e02554b4e1 (DEFAULT_TABLE table=test-table [id=9ba289e88a584d43bb71ad2b31e07007]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:49:41.737993 25824 tablet_service.cc:1467] Processing CreateTablet for tablet 73d44206237a460bb2e0b3e02554b4e1 (DEFAULT_TABLE table=test-table [id=9ba289e88a584d43bb71ad2b31e07007]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:49:41.738229 26024 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73d44206237a460bb2e0b3e02554b4e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:49:41.738359 25824 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73d44206237a460bb2e0b3e02554b4e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:49:41.740252 26340 tablet_bootstrap.cc:492] T 73d44206237a460bb2e0b3e02554b4e1 P 87fd367889cc4d379ad9e432f8572051: Bootstrap starting.
I20260806 22:49:41.741217 26340 tablet_bootstrap.cc:654] T 73d44206237a460bb2e0b3e02554b4e1 P 87fd367889cc4d379ad9e432f8572051: Neither blocks nor log segments found. Creating new log.
I20260806 22:49:41.741346 26341 tablet_bootstrap.cc:492] T 73d44206237a460bb2e0b3e02554b4e1 P 91e4eeee79304b0da16d36864b9fcc1f: Bootstrap starting.
I20260806 22:49:41.741945 26343 tablet_bootstrap.cc:492] T 73d44206237a460bb2e0b3e02554b4e1 P fd1b2e1656e24421813074d3a7a3f6bb: Bootstrap starting.
I20260806 22:49:41.742048 26340 log.cc:824] T 73d44206237a460bb2e0b3e02554b4e1 P 87fd367889cc4d379ad9e432f8572051: Log is configured to *not* fsync() on all Append() calls
I20260806 22:49:41.742125 26341 tablet_bootstrap.cc:654] T 73d44206237a460bb2e0b3e02554b4e1 P 91e4eeee79304b0da16d36864b9fcc1f: Neither blocks nor log segments found. Creating new log.
I20260806 22:49:41.742770 26343 tablet_bootstrap.cc:654] T 73d44206237a460bb2e0b3e02554b4e1 P fd1b2e1656e24421813074d3a7a3f6bb: Neither blocks nor log segments found. Creating new log.
I20260806 22:49:41.742802 26341 log.cc:824] T 73d44206237a460bb2e0b3e02554b4e1 P 91e4eeee79304b0da16d36864b9fcc1f: Log is configured to *not* fsync() on all Append() calls
I20260806 22:49:41.742924 26340 tablet_bootstrap.cc:492] T 73d44206237a460bb2e0b3e02554b4e1 P 87fd367889cc4d379ad9e432f8572051: No bootstrap required, opened a new log
I20260806 22:49:41.743006 26340 ts_tablet_manager.cc:1397] T 73d44206237a460bb2e0b3e02554b4e1 P 87fd367889cc4d379ad9e432f8572051: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 22:49:41.743381 26343 log.cc:824] T 73d44206237a460bb2e0b3e02554b4e1 P fd1b2e1656e24421813074d3a7a3f6bb: Log is configured to *not* fsync() on all Append() calls
I20260806 22:49:41.743723 26341 tablet_bootstrap.cc:492] T 73d44206237a460bb2e0b3e02554b4e1 P 91e4eeee79304b0da16d36864b9fcc1f: No bootstrap required, opened a new log
I20260806 22:49:41.743795 26341 ts_tablet_manager.cc:1397] T 73d44206237a460bb2e0b3e02554b4e1 P 91e4eeee79304b0da16d36864b9fcc1f: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 22:49:41.744272 26343 tablet_bootstrap.cc:492] T 73d44206237a460bb2e0b3e02554b4e1 P fd1b2e1656e24421813074d3a7a3f6bb: No bootstrap required, opened a new log
I20260806 22:49:41.744336 26343 ts_tablet_manager.cc:1397] T 73d44206237a460bb2e0b3e02554b4e1 P fd1b2e1656e24421813074d3a7a3f6bb: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:49:41.744897 26340 raft_consensus.cc:348] T 73d44206237a460bb2e0b3e02554b4e1 P 87fd367889cc4d379ad9e432f8572051 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd1b2e1656e24421813074d3a7a3f6bb" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 43385 } } peers { permanent_uuid: "91e4eeee79304b0da16d36864b9fcc1f" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 38635 } } peers { permanent_uuid: "87fd367889cc4d379ad9e432f8572051" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 46713 } }
I20260806 22:49:41.745061 26340 raft_consensus.cc:374] T 73d44206237a460bb2e0b3e02554b4e1 P 87fd367889cc4d379ad9e432f8572051 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:49:41.745087 26340 raft_consensus.cc:729] T 73d44206237a460bb2e0b3e02554b4e1 P 87fd367889cc4d379ad9e432f8572051 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87fd367889cc4d379ad9e432f8572051, State: Initialized, Role: FOLLOWER
I20260806 22:49:41.745209 26340 consensus_queue.cc:260] T 73d44206237a460bb2e0b3e02554b4e1 P 87fd367889cc4d379ad9e432f8572051 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd1b2e1656e24421813074d3a7a3f6bb" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 43385 } } peers { permanent_uuid: "91e4eeee79304b0da16d36864b9fcc1f" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 38635 } } peers { permanent_uuid: "87fd367889cc4d379ad9e432f8572051" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 46713 } }
I20260806 22:49:41.745512 26317 heartbeater.cc:503] Master 127.6.40.126:32939 was elected leader, sending a full tablet report...
I20260806 22:49:41.745512 26340 ts_tablet_manager.cc:1428] T 73d44206237a460bb2e0b3e02554b4e1 P 87fd367889cc4d379ad9e432f8572051: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:49:41.745903 26341 raft_consensus.cc:348] T 73d44206237a460bb2e0b3e02554b4e1 P 91e4eeee79304b0da16d36864b9fcc1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd1b2e1656e24421813074d3a7a3f6bb" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 43385 } } peers { permanent_uuid: "91e4eeee79304b0da16d36864b9fcc1f" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 38635 } } peers { permanent_uuid: "87fd367889cc4d379ad9e432f8572051" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 46713 } }
I20260806 22:49:41.746073 26341 raft_consensus.cc:374] T 73d44206237a460bb2e0b3e02554b4e1 P 91e4eeee79304b0da16d36864b9fcc1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:49:41.746109 26341 raft_consensus.cc:729] T 73d44206237a460bb2e0b3e02554b4e1 P 91e4eeee79304b0da16d36864b9fcc1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91e4eeee79304b0da16d36864b9fcc1f, State: Initialized, Role: FOLLOWER
I20260806 22:49:41.746223 26341 consensus_queue.cc:260] T 73d44206237a460bb2e0b3e02554b4e1 P 91e4eeee79304b0da16d36864b9fcc1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd1b2e1656e24421813074d3a7a3f6bb" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 43385 } } peers { permanent_uuid: "91e4eeee79304b0da16d36864b9fcc1f" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 38635 } } peers { permanent_uuid: "87fd367889cc4d379ad9e432f8572051" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 46713 } }
I20260806 22:49:41.746347 26343 raft_consensus.cc:348] T 73d44206237a460bb2e0b3e02554b4e1 P fd1b2e1656e24421813074d3a7a3f6bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd1b2e1656e24421813074d3a7a3f6bb" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 43385 } } peers { permanent_uuid: "91e4eeee79304b0da16d36864b9fcc1f" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 38635 } } peers { permanent_uuid: "87fd367889cc4d379ad9e432f8572051" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 46713 } }
I20260806 22:49:41.746505 26343 raft_consensus.cc:374] T 73d44206237a460bb2e0b3e02554b4e1 P fd1b2e1656e24421813074d3a7a3f6bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:49:41.746500 26341 ts_tablet_manager.cc:1428] T 73d44206237a460bb2e0b3e02554b4e1 P 91e4eeee79304b0da16d36864b9fcc1f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:49:41.746538 26343 raft_consensus.cc:729] T 73d44206237a460bb2e0b3e02554b4e1 P fd1b2e1656e24421813074d3a7a3f6bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd1b2e1656e24421813074d3a7a3f6bb, State: Initialized, Role: FOLLOWER
I20260806 22:49:41.746630 26123 heartbeater.cc:503] Master 127.6.40.126:32939 was elected leader, sending a full tablet report...
I20260806 22:49:41.746855 26343 consensus_queue.cc:260] T 73d44206237a460bb2e0b3e02554b4e1 P fd1b2e1656e24421813074d3a7a3f6bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd1b2e1656e24421813074d3a7a3f6bb" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 43385 } } peers { permanent_uuid: "91e4eeee79304b0da16d36864b9fcc1f" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 38635 } } peers { permanent_uuid: "87fd367889cc4d379ad9e432f8572051" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 46713 } }
I20260806 22:49:41.747228 26343 ts_tablet_manager.cc:1428] T 73d44206237a460bb2e0b3e02554b4e1 P fd1b2e1656e24421813074d3a7a3f6bb: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:49:41.747378 25924 heartbeater.cc:503] Master 127.6.40.126:32939 was elected leader, sending a full tablet report...
W20260806 22:49:41.773579 26124 tablet.cc:2369] T 73d44206237a460bb2e0b3e02554b4e1 P 91e4eeee79304b0da16d36864b9fcc1f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:49:41.781522 26355 raft_consensus.cc:482] T 73d44206237a460bb2e0b3e02554b4e1 P 91e4eeee79304b0da16d36864b9fcc1f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:49:41.781687 26355 raft_consensus.cc:504] T 73d44206237a460bb2e0b3e02554b4e1 P 91e4eeee79304b0da16d36864b9fcc1f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd1b2e1656e24421813074d3a7a3f6bb" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 43385 } } peers { permanent_uuid: "91e4eeee79304b0da16d36864b9fcc1f" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 38635 } } peers { permanent_uuid: "87fd367889cc4d379ad9e432f8572051" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 46713 } }
I20260806 22:49:41.782056 26355 leader_election.cc:288] T 73d44206237a460bb2e0b3e02554b4e1 P 91e4eeee79304b0da16d36864b9fcc1f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fd1b2e1656e24421813074d3a7a3f6bb (127.6.40.65:43385), 87fd367889cc4d379ad9e432f8572051 (127.6.40.67:46713)
I20260806 22:49:41.785926 26337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:41.782351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.66:48549 (local address 127.6.40.65:43385)
0806 22:49:41.782407 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:41.782409 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:41.782419 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:49:41.782448 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:41.782449 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:41.782461 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:41.782525 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:49:41.782731 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:41.783514 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:41.784939 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:41.785552 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:41.785618 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:41.785629 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:41.785813 (+ 184us) server_negotiation.cc:299] Negotiation successful
0806 22:49:41.785853 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 22:49:41.785926 26339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:41.782457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.66:52471 (local address 127.6.40.67:46713)
0806 22:49:41.782533 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:41.782535 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:41.782712 (+ 177us) server_negotiation.cc:407] Connection header received
0806 22:49:41.782744 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:41.782745 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:41.782759 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:41.782817 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:49:41.782989 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:41.783662 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:41.784953 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:41.785536 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:41.785618 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:41.785628 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:41.785813 (+ 185us) server_negotiation.cc:299] Negotiation successful
0806 22:49:41.785854 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 22:49:41.785950 26125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:41.782232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.65:43385 (local address 127.6.40.66:48549)
0806 22:49:41.782302 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:41.782315 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:41.782386 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:41.782533 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:41.782535 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:41.782536 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:49:41.782646 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:41.782648 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:41.783603 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:41.783604 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:41.784859 (+ 1255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:41.784862 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:41.785639 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:41.785640 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:41.785731 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:41.785733 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 22:49:41.785794 (+ 61us) client_negotiation.cc:240] Negotiation successful
0806 22:49:41.785842 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44,"mutex_wait_us":6}
I20260806 22:49:41.785957 26357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:41.782391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.67:46713 (local address 127.6.40.66:52471)
0806 22:49:41.782613 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:41.782625 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:41.782685 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:41.782828 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:41.782829 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:41.782830 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:49:41.782887 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:41.782888 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:41.783732 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:41.783733 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:41.784860 (+ 1127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:41.784862 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:41.785639 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:41.785640 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:41.785725 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:41.785730 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:49:41.785794 (+ 64us) client_negotiation.cc:240] Negotiation successful
0806 22:49:41.785841 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203,"mutex_wait_us":6,"thread_start_us":147,"threads_started":1}
I20260806 22:49:41.786374 25854 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73d44206237a460bb2e0b3e02554b4e1" candidate_uuid: "91e4eeee79304b0da16d36864b9fcc1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd1b2e1656e24421813074d3a7a3f6bb" is_pre_election: true
I20260806 22:49:41.786515 25854 raft_consensus.cc:2454] T 73d44206237a460bb2e0b3e02554b4e1 P fd1b2e1656e24421813074d3a7a3f6bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91e4eeee79304b0da16d36864b9fcc1f in term 0.
I20260806 22:49:41.786458 26248 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73d44206237a460bb2e0b3e02554b4e1" candidate_uuid: "91e4eeee79304b0da16d36864b9fcc1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87fd367889cc4d379ad9e432f8572051" is_pre_election: true
I20260806 22:49:41.786674 26248 raft_consensus.cc:2454] T 73d44206237a460bb2e0b3e02554b4e1 P 87fd367889cc4d379ad9e432f8572051 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91e4eeee79304b0da16d36864b9fcc1f in term 0.
I20260806 22:49:41.786741 25957 leader_election.cc:302] T 73d44206237a460bb2e0b3e02554b4e1 P 91e4eeee79304b0da16d36864b9fcc1f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91e4eeee79304b0da16d36864b9fcc1f, fd1b2e1656e24421813074d3a7a3f6bb; no voters:
I20260806 22:49:41.786865 26355 raft_consensus.cc:2781] T 73d44206237a460bb2e0b3e02554b4e1 P 91e4eeee79304b0da16d36864b9fcc1f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:49:41.786938 26355 raft_consensus.cc:482] T 73d44206237a460bb2e0b3e02554b4e1 P 91e4eeee79304b0da16d36864b9fcc1f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:49:41.786967 26355 raft_consensus.cc:3037] T 73d44206237a460bb2e0b3e02554b4e1 P 91e4eeee79304b0da16d36864b9fcc1f [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:49:41.787756 26355 raft_consensus.cc:504] T 73d44206237a460bb2e0b3e02554b4e1 P 91e4eeee79304b0da16d36864b9fcc1f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd1b2e1656e24421813074d3a7a3f6bb" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 43385 } } peers { permanent_uuid: "91e4eeee79304b0da16d36864b9fcc1f" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 38635 } } peers { permanent_uuid: "87fd367889cc4d379ad9e432f8572051" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 46713 } }
I20260806 22:49:41.787892 26355 leader_election.cc:288] T 73d44206237a460bb2e0b3e02554b4e1 P 91e4eeee79304b0da16d36864b9fcc1f [CANDIDATE]: Term 1 election: Requested vote from peers fd1b2e1656e24421813074d3a7a3f6bb (127.6.40.65:43385), 87fd367889cc4d379ad9e432f8572051 (127.6.40.67:46713)
I20260806 22:49:41.788092 25854 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73d44206237a460bb2e0b3e02554b4e1" candidate_uuid: "91e4eeee79304b0da16d36864b9fcc1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd1b2e1656e24421813074d3a7a3f6bb"
I20260806 22:49:41.788101 26248 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73d44206237a460bb2e0b3e02554b4e1" candidate_uuid: "91e4eeee79304b0da16d36864b9fcc1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87fd367889cc4d379ad9e432f8572051"
I20260806 22:49:41.788182 26248 raft_consensus.cc:3037] T 73d44206237a460bb2e0b3e02554b4e1 P 87fd367889cc4d379ad9e432f8572051 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:49:41.788185 25854 raft_consensus.cc:3037] T 73d44206237a460bb2e0b3e02554b4e1 P fd1b2e1656e24421813074d3a7a3f6bb [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:49:41.789197 26248 raft_consensus.cc:2454] T 73d44206237a460bb2e0b3e02554b4e1 P 87fd367889cc4d379ad9e432f8572051 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91e4eeee79304b0da16d36864b9fcc1f in term 1.
I20260806 22:49:41.789197 25854 raft_consensus.cc:2454] T 73d44206237a460bb2e0b3e02554b4e1 P fd1b2e1656e24421813074d3a7a3f6bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91e4eeee79304b0da16d36864b9fcc1f in term 1.
I20260806 22:49:41.789417 25957 leader_election.cc:302] T 73d44206237a460bb2e0b3e02554b4e1 P 91e4eeee79304b0da16d36864b9fcc1f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 87fd367889cc4d379ad9e432f8572051, 91e4eeee79304b0da16d36864b9fcc1f; no voters:
I20260806 22:49:41.789512 26355 raft_consensus.cc:2781] T 73d44206237a460bb2e0b3e02554b4e1 P 91e4eeee79304b0da16d36864b9fcc1f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:49:41.789732 26355 raft_consensus.cc:686] T 73d44206237a460bb2e0b3e02554b4e1 P 91e4eeee79304b0da16d36864b9fcc1f [term 1 LEADER]: Becoming Leader. State: Replica: 91e4eeee79304b0da16d36864b9fcc1f, State: Running, Role: LEADER
I20260806 22:49:41.789845 26355 consensus_queue.cc:237] T 73d44206237a460bb2e0b3e02554b4e1 P 91e4eeee79304b0da16d36864b9fcc1f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd1b2e1656e24421813074d3a7a3f6bb" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 43385 } } peers { permanent_uuid: "91e4eeee79304b0da16d36864b9fcc1f" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 38635 } } peers { permanent_uuid: "87fd367889cc4d379ad9e432f8572051" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 46713 } }
I20260806 22:49:41.790695 25652 catalog_manager.cc:5495] T 73d44206237a460bb2e0b3e02554b4e1 P 91e4eeee79304b0da16d36864b9fcc1f reported cstate change: term changed from 0 to 1, leader changed from <none> to 91e4eeee79304b0da16d36864b9fcc1f (127.6.40.66). New cstate: current_term: 1 leader_uuid: "91e4eeee79304b0da16d36864b9fcc1f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd1b2e1656e24421813074d3a7a3f6bb" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 43385 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91e4eeee79304b0da16d36864b9fcc1f" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 38635 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "87fd367889cc4d379ad9e432f8572051" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 46713 } health_report { overall_health: UNKNOWN } } }
I20260806 22:49:41.797555 26336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:41.795068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49004 (local address 127.6.40.66:38635)
0806 22:49:41.795119 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:41.795121 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:41.795133 (+ 12us) server_negotiation.cc:407] Connection header received
0806 22:49:41.795168 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:41.795169 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:41.795196 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:41.795273 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:49:41.795403 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:41.795865 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:41.796640 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:41.796832 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:41.796954 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:41.797124 (+ 170us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:49:41.797223 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:49:41.797312 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:41.797395 (+ 83us) server_negotiation.cc:299] Negotiation successful
0806 22:49:41.797418 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 22:49:41.799273 26248 raft_consensus.cc:1261] T 73d44206237a460bb2e0b3e02554b4e1 P 87fd367889cc4d379ad9e432f8572051 [term 1 FOLLOWER]: Refusing update from remote peer 91e4eeee79304b0da16d36864b9fcc1f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:49:41.799309 25854 raft_consensus.cc:1261] T 73d44206237a460bb2e0b3e02554b4e1 P fd1b2e1656e24421813074d3a7a3f6bb [term 1 FOLLOWER]: Refusing update from remote peer 91e4eeee79304b0da16d36864b9fcc1f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:49:41.799731 26361 consensus_queue.cc:1035] T 73d44206237a460bb2e0b3e02554b4e1 P 91e4eeee79304b0da16d36864b9fcc1f [LEADER]: Connected to new peer: Peer: permanent_uuid: "87fd367889cc4d379ad9e432f8572051" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 46713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:49:41.799857 26355 consensus_queue.cc:1035] T 73d44206237a460bb2e0b3e02554b4e1 P 91e4eeee79304b0da16d36864b9fcc1f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd1b2e1656e24421813074d3a7a3f6bb" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 43385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:49:41.803761 26368 mvcc.cc:205] Tried to move back new op lower bound from 7315687759047147520 to 7315687759011819520. Current Snapshot: MvccSnapshot[applied={T|T < 7315687759047147520 or (T in {7315687759047147520})}]
I20260806 22:49:41.803962 26367 mvcc.cc:205] Tried to move back new op lower bound from 7315687759047147520 to 7315687759011819520. Current Snapshot: MvccSnapshot[applied={T|T < 7315687759047147520 or (T in {7315687759047147520})}]
I20260806 22:49:41.804172 26370 mvcc.cc:205] Tried to move back new op lower bound from 7315687759047147520 to 7315687759011819520. Current Snapshot: MvccSnapshot[applied={T|T < 7315687759047147520 or (T in {7315687759047147520})}]
I20260806 22:49:41.805487 25651 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51238:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:49:41.805585 25651 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51238:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:49:41.806396 25651 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b9e1eb63b32f4e9ea1c7f100f2d38d4c: Sending DeleteTablet for 3 replicas of tablet 73d44206237a460bb2e0b3e02554b4e1
I20260806 22:49:41.806769 26024 tablet_service.cc:1514] Processing DeleteTablet for tablet 73d44206237a460bb2e0b3e02554b4e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:49:41 UTC) from {username='kudu', principal='kudu/127.6.40.126@KRBTEST.COM'} at 127.0.0.1:48992
I20260806 22:49:41.806890 26220 tablet_service.cc:1514] Processing DeleteTablet for tablet 73d44206237a460bb2e0b3e02554b4e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:49:41 UTC) from {username='kudu', principal='kudu/127.6.40.126@KRBTEST.COM'} at 127.0.0.1:39304
I20260806 22:49:41.807030 26383 tablet_replica.cc:330] T 73d44206237a460bb2e0b3e02554b4e1 P 91e4eeee79304b0da16d36864b9fcc1f: stopping tablet replica
I20260806 22:49:41.807032 25824 tablet_service.cc:1514] Processing DeleteTablet for tablet 73d44206237a460bb2e0b3e02554b4e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:49:41 UTC) from {username='kudu', principal='kudu/127.6.40.126@KRBTEST.COM'} at 127.0.0.1:36554
I20260806 22:49:41.807127 26383 raft_consensus.cc:2229] T 73d44206237a460bb2e0b3e02554b4e1 P 91e4eeee79304b0da16d36864b9fcc1f [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:49:41.807214 26383 raft_consensus.cc:2258] T 73d44206237a460bb2e0b3e02554b4e1 P 91e4eeee79304b0da16d36864b9fcc1f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:49:41.807231 26385 tablet_replica.cc:330] T 73d44206237a460bb2e0b3e02554b4e1 P fd1b2e1656e24421813074d3a7a3f6bb: stopping tablet replica
I20260806 22:49:41.807255 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 25734
I20260806 22:49:41.822839 26383 ts_tablet_manager.cc:1905] T 73d44206237a460bb2e0b3e02554b4e1 P 91e4eeee79304b0da16d36864b9fcc1f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:49:41.823565 26384 tablet_replica.cc:330] T 73d44206237a460bb2e0b3e02554b4e1 P 87fd367889cc4d379ad9e432f8572051: stopping tablet replica
I20260806 22:49:41.823773 26384 raft_consensus.cc:2229] T 73d44206237a460bb2e0b3e02554b4e1 P 87fd367889cc4d379ad9e432f8572051 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:49:41.823958 26384 raft_consensus.cc:2258] T 73d44206237a460bb2e0b3e02554b4e1 P 87fd367889cc4d379ad9e432f8572051 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:49:41.824604 26384 ts_tablet_manager.cc:1905] T 73d44206237a460bb2e0b3e02554b4e1 P 87fd367889cc4d379ad9e432f8572051: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:49:41.824955 26383 ts_tablet_manager.cc:1918] T 73d44206237a460bb2e0b3e02554b4e1 P 91e4eeee79304b0da16d36864b9fcc1f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:49:41.825095 26383 log.cc:1196] T 73d44206237a460bb2e0b3e02554b4e1 P 91e4eeee79304b0da16d36864b9fcc1f: Deleting WAL directory at /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-1/wal/wals/73d44206237a460bb2e0b3e02554b4e1
I20260806 22:49:41.825484 26383 ts_tablet_manager.cc:1939] T 73d44206237a460bb2e0b3e02554b4e1 P 91e4eeee79304b0da16d36864b9fcc1f: Deleting consensus metadata
I20260806 22:49:41.826756 26384 ts_tablet_manager.cc:1918] T 73d44206237a460bb2e0b3e02554b4e1 P 87fd367889cc4d379ad9e432f8572051: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:49:41.826844 26384 log.cc:1196] T 73d44206237a460bb2e0b3e02554b4e1 P 87fd367889cc4d379ad9e432f8572051: Deleting WAL directory at /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-2/wal/wals/73d44206237a460bb2e0b3e02554b4e1
I20260806 22:49:41.827173 26384 ts_tablet_manager.cc:1939] T 73d44206237a460bb2e0b3e02554b4e1 P 87fd367889cc4d379ad9e432f8572051: Deleting consensus metadata
I20260806 22:49:41.827667 25630 catalog_manager.cc:4841] TS 91e4eeee79304b0da16d36864b9fcc1f (127.6.40.66:38635): tablet 73d44206237a460bb2e0b3e02554b4e1 (table test-table [id=9ba289e88a584d43bb71ad2b31e07007]) successfully deleted
I20260806 22:49:41.827669 25627 catalog_manager.cc:4841] TS 87fd367889cc4d379ad9e432f8572051 (127.6.40.67:46713): tablet 73d44206237a460bb2e0b3e02554b4e1 (table test-table [id=9ba289e88a584d43bb71ad2b31e07007]) successfully deleted
W20260806 22:49:41.829202 25627 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.40.65:43385 (error 108)
I20260806 22:49:41.829435 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 25937
I20260806 22:49:41.832196 26333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:41.829677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.65:43385 (local address 127.0.0.1:36560)
0806 22:49:41.832121 (+ 2444us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:41.832164 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.65:43385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2422}
W20260806 22:49:41.835039 25627 catalog_manager.cc:4584] TS fd1b2e1656e24421813074d3a7a3f6bb (127.6.40.65:43385): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 73d44206237a460bb2e0b3e02554b4e1: Network error: Client connection negotiation failed: client connection to 127.6.40.65:43385: connect: Connection refused (error 111)
I20260806 22:49:41.839797 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 26129
I20260806 22:49:41.852214 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 25604
I20260806 22:49:41.860949 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.126:32939
--webserver_interface=127.6.40.126
--webserver_port=45679
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab
--principal=kudu/127.6.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:42369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.40.126:32939
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:49:41.948316 26388 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:49:41.948624 26388 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:49:41.948665 26388 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:49:41.949851 26388 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:49:41.949920 26388 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:49:41.949944 26388 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:49:41.949966 26388 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:49:41.949996 26388 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:49:41.951476 26388 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:42369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.6.40.126:32939
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab
--principal=kudu/127.6.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.126:32939
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.40.126
--webserver_port=45679
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26388
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:49:41.951761 26388 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:49:41.951995 26388 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:49:41.961753 26396 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:41.961702 26399 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:41.961750 26395 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:41.962069 26388 server_base.cc:1034] running on GCE node
I20260806 22:49:41.962345 26388 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:49:41.962633 26388 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:49:41.963785 26388 hybrid_clock.cc:648] HybridClock initialized: now 1786056581963747 us; error 50 us; skew 500 ppm
Aug 06 22:49:41 dist-test-slave-zh2d krb5kdc[25586](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056581, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:49:41.966547 26388 init.cc:377] Logged in from keytab as kudu/127.6.40.126@KRBTEST.COM (short username kudu)
I20260806 22:49:41.967103 26388 webserver.cc:460] Webserver started at http://127.6.40.126:45679/ using document root <none> and password file <none>
I20260806 22:49:41.967396 26388 fs_manager.cc:359] Metadata directory not provided
I20260806 22:49:41.967489 26388 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:49:41.969017 26388 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 22:49:41.969707 26409 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:41.969882 26388 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:49:41.970005 26388 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/master-0/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/master-0/wal
uuid: "b9e1eb63b32f4e9ea1c7f100f2d38d4c"
format_stamp: "Formatted at 2026-08-06 22:49:41 on dist-test-slave-zh2d"
server_key: "96801279d1363050246285ee624aa72c"
server_key_iv: "501ed5420d49cdc58d4f18edeeeed1a7"
server_key_version: "encryptionkey@0"
I20260806 22:49:41.970439 26388 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:49:41.984318 26388 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:49:41.986279 26388 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:49:41.986449 26388 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:49:41.991549 26388 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.126:32939
I20260806 22:49:41.991577 26479 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.126:32939 every 8 connection(s)
I20260806 22:49:41.991995 26388 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
I20260806 22:49:41.993731 26480 sys_catalog.cc:263] Verifying existing consensus state
I20260806 22:49:41.994421 26480 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9e1eb63b32f4e9ea1c7f100f2d38d4c: Bootstrap starting.
I20260806 22:49:41.997144 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 26388
I20260806 22:49:41.997506 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.65:43385
--local_ip_for_outbound_sockets=127.6.40.65
--tserver_master_addrs=127.6.40.126:32939
--webserver_port=38705
--webserver_interface=127.6.40.65
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.65.keytab
--principal=kudu/127.6.40.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:42369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:49:42.005182 26480 log.cc:824] T 00000000000000000000000000000000 P b9e1eb63b32f4e9ea1c7f100f2d38d4c: Log is configured to *not* fsync() on all Append() calls
I20260806 22:49:42.007213 26480 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9e1eb63b32f4e9ea1c7f100f2d38d4c: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 22:49:42.007561 26480 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9e1eb63b32f4e9ea1c7f100f2d38d4c: Bootstrap complete.
I20260806 22:49:42.010097 26480 raft_consensus.cc:348] T 00000000000000000000000000000000 P b9e1eb63b32f4e9ea1c7f100f2d38d4c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9e1eb63b32f4e9ea1c7f100f2d38d4c" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 32939 } }
I20260806 22:49:42.010330 26480 raft_consensus.cc:729] T 00000000000000000000000000000000 P b9e1eb63b32f4e9ea1c7f100f2d38d4c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9e1eb63b32f4e9ea1c7f100f2d38d4c, State: Initialized, Role: FOLLOWER
I20260806 22:49:42.010509 26480 consensus_queue.cc:260] T 00000000000000000000000000000000 P b9e1eb63b32f4e9ea1c7f100f2d38d4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9e1eb63b32f4e9ea1c7f100f2d38d4c" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 32939 } }
I20260806 22:49:42.010571 26480 raft_consensus.cc:388] T 00000000000000000000000000000000 P b9e1eb63b32f4e9ea1c7f100f2d38d4c [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:49:42.010629 26480 raft_consensus.cc:482] T 00000000000000000000000000000000 P b9e1eb63b32f4e9ea1c7f100f2d38d4c [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:49:42.010679 26480 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b9e1eb63b32f4e9ea1c7f100f2d38d4c [term 1 FOLLOWER]: Advancing to term 2
I20260806 22:49:42.011747 26480 raft_consensus.cc:504] T 00000000000000000000000000000000 P b9e1eb63b32f4e9ea1c7f100f2d38d4c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9e1eb63b32f4e9ea1c7f100f2d38d4c" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 32939 } }
I20260806 22:49:42.011862 26480 leader_election.cc:302] T 00000000000000000000000000000000 P b9e1eb63b32f4e9ea1c7f100f2d38d4c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b9e1eb63b32f4e9ea1c7f100f2d38d4c; no voters:
I20260806 22:49:42.012066 26480 leader_election.cc:288] T 00000000000000000000000000000000 P b9e1eb63b32f4e9ea1c7f100f2d38d4c [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 22:49:42.012127 26485 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b9e1eb63b32f4e9ea1c7f100f2d38d4c [term 2 FOLLOWER]: Leader election won for term 2
I20260806 22:49:42.012275 26485 raft_consensus.cc:686] T 00000000000000000000000000000000 P b9e1eb63b32f4e9ea1c7f100f2d38d4c [term 2 LEADER]: Becoming Leader. State: Replica: b9e1eb63b32f4e9ea1c7f100f2d38d4c, State: Running, Role: LEADER
I20260806 22:49:42.012351 26480 sys_catalog.cc:564] T 00000000000000000000000000000000 P b9e1eb63b32f4e9ea1c7f100f2d38d4c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:49:42.012456 26485 consensus_queue.cc:237] T 00000000000000000000000000000000 P b9e1eb63b32f4e9ea1c7f100f2d38d4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9e1eb63b32f4e9ea1c7f100f2d38d4c" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 32939 } }
I20260806 22:49:42.012804 26488 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9e1eb63b32f4e9ea1c7f100f2d38d4c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "b9e1eb63b32f4e9ea1c7f100f2d38d4c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9e1eb63b32f4e9ea1c7f100f2d38d4c" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 32939 } } }
I20260806 22:49:42.012895 26488 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9e1eb63b32f4e9ea1c7f100f2d38d4c [sys.catalog]: This master's current role is: LEADER
I20260806 22:49:42.013305 26489 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9e1eb63b32f4e9ea1c7f100f2d38d4c [sys.catalog]: SysCatalogTable state changed. Reason: New leader b9e1eb63b32f4e9ea1c7f100f2d38d4c. Latest consensus state: current_term: 2 leader_uuid: "b9e1eb63b32f4e9ea1c7f100f2d38d4c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9e1eb63b32f4e9ea1c7f100f2d38d4c" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 32939 } } }
I20260806 22:49:42.013401 26489 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9e1eb63b32f4e9ea1c7f100f2d38d4c [sys.catalog]: This master's current role is: LEADER
I20260806 22:49:42.013557 26497 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:49:42.014394 26497 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:49:42.014503 26497 catalog_manager.cc:1259] Loaded cluster ID: bf2abec72c424d718a8be5897a9eb258
I20260806 22:49:42.014581 26497 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:49:42.014891 26497 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260806 22:49:42.019873 26497 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260806 22:49:42.022339 26497 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:49:42.022562 26497 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b9e1eb63b32f4e9ea1c7f100f2d38d4c: Loaded TSK: 0
I20260806 22:49:42.022768 26497 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 22:49:42.086289 26482 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:49:42.086493 26482 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:49:42.086516 26482 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:49:42.087666 26482 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:49:42.087706 26482 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:49:42.087733 26482 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.65
I20260806 22:49:42.089229 26482 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:42369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.65.keytab
--principal=kudu/127.6.40.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.65:43385
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.40.65
--webserver_port=38705
--tserver_master_addrs=127.6.40.126:32939
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26482
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.65
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:49:42.089464 26482 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:49:42.089730 26482 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:49:42.095129 26523 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:42.095279 26519 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:42.095129 26517 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:42.095391 26482 server_base.cc:1034] running on GCE node
I20260806 22:49:42.096793 26482 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:49:42.097106 26482 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:49:42.098299 26482 hybrid_clock.cc:648] HybridClock initialized: now 1786056582098268 us; error 46 us; skew 500 ppm
Aug 06 22:49:42 dist-test-slave-zh2d krb5kdc[25586](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056582, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:49:42.102470 26482 init.cc:377] Logged in from keytab as kudu/127.6.40.65@KRBTEST.COM (short username kudu)
I20260806 22:49:42.103009 26482 webserver.cc:460] Webserver started at http://127.6.40.65:38705/ using document root <none> and password file <none>
I20260806 22:49:42.103289 26482 fs_manager.cc:359] Metadata directory not provided
I20260806 22:49:42.103371 26482 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:49:42.105202 26482 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:49:42.106073 26531 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:42.106298 26482 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:49:42.106411 26482 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-0/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-0/wal
uuid: "fd1b2e1656e24421813074d3a7a3f6bb"
format_stamp: "Formatted at 2026-08-06 22:49:41 on dist-test-slave-zh2d"
server_key: "1debb8747148c3197bcef363e7b0d86f"
server_key_iv: "5852b2f46bb327f4ea1ba328b32abaad"
server_key_version: "encryptionkey@0"
I20260806 22:49:42.106772 26482 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:49:42.121165 26482 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:49:42.123507 26482 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:49:42.123694 26482 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:49:42.123942 26482 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:49:42.124414 26539 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 22:49:42.126554 26482 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 22:49:42.126626 26482 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260806 22:49:42.126670 26482 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 22:49:42.127351 26482 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 22:49:42.127411 26482 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260806 22:49:42.127485 26539 tablet_bootstrap.cc:492] T 73d44206237a460bb2e0b3e02554b4e1 P fd1b2e1656e24421813074d3a7a3f6bb: Bootstrap starting.
I20260806 22:49:42.135218 26482 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.65:43385
I20260806 22:49:42.135622 26482 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
I20260806 22:49:42.139577 26539 log.cc:824] T 73d44206237a460bb2e0b3e02554b4e1 P fd1b2e1656e24421813074d3a7a3f6bb: Log is configured to *not* fsync() on all Append() calls
I20260806 22:49:42.140230 26698 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.65:43385 every 8 connection(s)
I20260806 22:49:42.142764 26539 tablet_bootstrap.cc:492] T 73d44206237a460bb2e0b3e02554b4e1 P fd1b2e1656e24421813074d3a7a3f6bb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 22:49:42.143090 26539 tablet_bootstrap.cc:492] T 73d44206237a460bb2e0b3e02554b4e1 P fd1b2e1656e24421813074d3a7a3f6bb: Bootstrap complete.
I20260806 22:49:42.144047 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 26482
I20260806 22:49:42.144325 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.66:38635
--local_ip_for_outbound_sockets=127.6.40.66
--tserver_master_addrs=127.6.40.126:32939
--webserver_port=44203
--webserver_interface=127.6.40.66
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.66.keytab
--principal=kudu/127.6.40.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:42369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:49:42.144505 26539 ts_tablet_manager.cc:1397] T 73d44206237a460bb2e0b3e02554b4e1 P fd1b2e1656e24421813074d3a7a3f6bb: Time spent bootstrapping tablet: real 0.017s user 0.005s sys 0.009s
I20260806 22:49:42.146061 26539 raft_consensus.cc:348] T 73d44206237a460bb2e0b3e02554b4e1 P fd1b2e1656e24421813074d3a7a3f6bb [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd1b2e1656e24421813074d3a7a3f6bb" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 43385 } } peers { permanent_uuid: "91e4eeee79304b0da16d36864b9fcc1f" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 38635 } } peers { permanent_uuid: "87fd367889cc4d379ad9e432f8572051" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 46713 } }
I20260806 22:49:42.146466 26539 raft_consensus.cc:729] T 73d44206237a460bb2e0b3e02554b4e1 P fd1b2e1656e24421813074d3a7a3f6bb [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd1b2e1656e24421813074d3a7a3f6bb, State: Initialized, Role: FOLLOWER
I20260806 22:49:42.146682 26539 consensus_queue.cc:260] T 73d44206237a460bb2e0b3e02554b4e1 P fd1b2e1656e24421813074d3a7a3f6bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd1b2e1656e24421813074d3a7a3f6bb" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 43385 } } peers { permanent_uuid: "91e4eeee79304b0da16d36864b9fcc1f" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 38635 } } peers { permanent_uuid: "87fd367889cc4d379ad9e432f8572051" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 46713 } }
I20260806 22:49:42.147047 26539 ts_tablet_manager.cc:1428] T 73d44206237a460bb2e0b3e02554b4e1 P fd1b2e1656e24421813074d3a7a3f6bb: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
Aug 06 22:49:42 dist-test-slave-zh2d krb5kdc[25586](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056582, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.65@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
I20260806 22:49:42.155426 26706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:42.140681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.65:54757 (local address 127.6.40.126:32939)
0806 22:49:42.141030 (+ 349us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:42.141038 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:42.141251 (+ 213us) server_negotiation.cc:407] Connection header received
0806 22:49:42.141895 (+ 644us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:42.141901 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:42.141951 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:42.142082 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:42.142745 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:42.143782 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:42.147550 (+ 3768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:42.148169 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:42.148281 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:42.151198 (+ 2917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:42.151203 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:42.151203 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:42.151216 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:42.153322 (+ 2106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:42.153889 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:42.153892 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:42.153893 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:42.154017 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:42.154339 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:42.154340 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:42.154340 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:42.154638 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:42.154727 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:42.155111 (+ 384us) server_negotiation.cc:299] Negotiation successful
0806 22:49:42.155245 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":300,"thread_start_us":86,"threads_started":1}
I20260806 22:49:42.155516 26705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:42.140438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:32939 (local address 127.6.40.65:54757)
0806 22:49:42.141098 (+ 660us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:42.141132 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:42.141667 (+ 535us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:42.142129 (+ 462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:42.142134 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:42.142419 (+ 285us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:49:42.142622 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:42.142628 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:42.146213 (+ 3585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:42.146215 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:42.147439 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:42.147443 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:42.148289 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:42.148290 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:42.148429 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:42.149136 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:49:42.149146 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:49:42.151081 (+ 1935us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:49:42.153415 (+ 2334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:42.153423 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:42.153426 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:42.153789 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:42.154106 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:42.154107 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:42.154107 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:42.154258 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:42.154723 (+ 465us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:49:42.154732 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:49:42.154909 (+ 177us) client_negotiation.cc:769] Sending connection context
0806 22:49:42.155040 (+ 131us) client_negotiation.cc:240] Negotiation successful
0806 22:49:42.155237 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":626,"thread_start_us":276,"threads_started":1}
I20260806 22:49:42.157218 26700 heartbeater.cc:347] Connected to a master server at 127.6.40.126:32939
I20260806 22:49:42.157321 26700 heartbeater.cc:464] Registering TS with master...
I20260806 22:49:42.157505 26700 heartbeater.cc:511] Master 127.6.40.126:32939 requested a full tablet report, sending...
I20260806 22:49:42.158329 26430 ts_manager.cc:194] Registered new tserver with Master: fd1b2e1656e24421813074d3a7a3f6bb (127.6.40.65:43385)
I20260806 22:49:42.159435 26430 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.40.65@KRBTEST.COM'} at 127.6.40.65:54757
I20260806 22:49:42.160404 26700 heartbeater.cc:503] Master 127.6.40.126:32939 was elected leader, sending a full tablet report...
Aug 06 22:49:42 dist-test-slave-zh2d krb5kdc[25586](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056581, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.126@KRBTEST.COM for kudu/127.6.40.65@KRBTEST.COM
I20260806 22:49:42.167457 26710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:42.158922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.65:43385 (local address 127.0.0.1:36568)
0806 22:49:42.159224 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:42.159261 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:42.159351 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:42.159611 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:42.159614 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:42.159967 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:49:42.160014 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:42.160016 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:42.161158 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:42.161160 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:42.161760 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:42.161763 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:42.162271 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:42.162272 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:42.162352 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:42.162830 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:49:42.162834 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:49:42.164321 (+ 1487us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:49:42.166399 (+ 2078us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:42.166405 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:42.166406 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:42.166592 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:42.166809 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:42.166810 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:42.166810 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:42.166862 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:42.167153 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:49:42.167159 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:49:42.167198 (+ 39us) client_negotiation.cc:769] Sending connection context
0806 22:49:42.167259 (+ 61us) client_negotiation.cc:240] Negotiation successful
0806 22:49:42.167381 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":91,"threads_started":1}
I20260806 22:49:42.167523 26711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:42.159002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36568 (local address 127.6.40.65:43385)
0806 22:49:42.159153 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:42.159157 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:42.159316 (+ 159us) server_negotiation.cc:407] Connection header received
0806 22:49:42.159452 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:42.159454 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:42.159470 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:42.159515 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:42.160107 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:42.161079 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:42.161857 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:42.162176 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:42.162267 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:42.164416 (+ 2149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:42.164424 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:42.164424 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:42.164432 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:42.166309 (+ 1877us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:42.166679 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:42.166682 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:42.166682 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:42.166745 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:42.166922 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:42.166922 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:42.166923 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:42.167089 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:42.167170 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:42.167316 (+ 146us) server_negotiation.cc:299] Negotiation successful
0806 22:49:42.167414 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":78,"threads_started":1}
I20260806 22:49:42.168103 26584 tablet_service.cc:1514] Processing DeleteTablet for tablet 73d44206237a460bb2e0b3e02554b4e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:49:41 UTC) from {username='kudu', principal='kudu/127.6.40.126@KRBTEST.COM'} at 127.0.0.1:36568
I20260806 22:49:42.168756 26713 tablet_replica.cc:330] stopping tablet replica
I20260806 22:49:42.168915 26713 raft_consensus.cc:2229] T 73d44206237a460bb2e0b3e02554b4e1 P fd1b2e1656e24421813074d3a7a3f6bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:49:42.168999 26713 raft_consensus.cc:2258] T 73d44206237a460bb2e0b3e02554b4e1 P fd1b2e1656e24421813074d3a7a3f6bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:49:42.169361 26713 ts_tablet_manager.cc:1905] T 73d44206237a460bb2e0b3e02554b4e1 P fd1b2e1656e24421813074d3a7a3f6bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:49:42.171416 26713 ts_tablet_manager.cc:1918] T 73d44206237a460bb2e0b3e02554b4e1 P fd1b2e1656e24421813074d3a7a3f6bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:49:42.171506 26713 log.cc:1196] T 73d44206237a460bb2e0b3e02554b4e1 P fd1b2e1656e24421813074d3a7a3f6bb: Deleting WAL directory at /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-0/wal/wals/73d44206237a460bb2e0b3e02554b4e1
I20260806 22:49:42.171726 26713 ts_tablet_manager.cc:1939] T 73d44206237a460bb2e0b3e02554b4e1 P fd1b2e1656e24421813074d3a7a3f6bb: Deleting consensus metadata
I20260806 22:49:42.172256 26410 catalog_manager.cc:4841] TS fd1b2e1656e24421813074d3a7a3f6bb (127.6.40.65:43385): tablet 73d44206237a460bb2e0b3e02554b4e1 (table test-table [id=9ba289e88a584d43bb71ad2b31e07007]) successfully deleted
W20260806 22:49:42.235327 26707 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:49:42.235532 26707 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:49:42.235556 26707 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:49:42.236655 26707 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:49:42.236692 26707 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:49:42.236718 26707 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.66
I20260806 22:49:42.238312 26707 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:42369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.66.keytab
--principal=kudu/127.6.40.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.66:38635
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.40.66
--webserver_port=44203
--tserver_master_addrs=127.6.40.126:32939
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26707
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.66
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:49:42.238505 26707 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:49:42.238684 26707 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:49:42.244160 26722 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:42.244154 26724 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:42.244155 26720 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:42.244176 26707 server_base.cc:1034] running on GCE node
I20260806 22:49:42.244580 26707 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:49:42.244872 26707 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:49:42.246038 26707 hybrid_clock.cc:648] HybridClock initialized: now 1786056582246011 us; error 34 us; skew 500 ppm
Aug 06 22:49:42 dist-test-slave-zh2d krb5kdc[25586](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056582, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:49:42.248659 26707 init.cc:377] Logged in from keytab as kudu/127.6.40.66@KRBTEST.COM (short username kudu)
I20260806 22:49:42.249200 26707 webserver.cc:460] Webserver started at http://127.6.40.66:44203/ using document root <none> and password file <none>
I20260806 22:49:42.249435 26707 fs_manager.cc:359] Metadata directory not provided
I20260806 22:49:42.249521 26707 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:49:42.250975 26707 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:49:42.251658 26735 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:42.251854 26707 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:49:42.251955 26707 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-1/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-1/wal
uuid: "91e4eeee79304b0da16d36864b9fcc1f"
format_stamp: "Formatted at 2026-08-06 22:49:41 on dist-test-slave-zh2d"
server_key: "9319cfeecf264e9c4274d57891c2e5a9"
server_key_iv: "23407928f59f72f10f6975735c620a85"
server_key_version: "encryptionkey@0"
I20260806 22:49:42.252324 26707 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:49:42.271178 26707 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:49:42.273759 26707 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:49:42.273933 26707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:49:42.274243 26707 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:49:42.274581 26707 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:49:42.274637 26707 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:42.274681 26707 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:49:42.274705 26707 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:42.280671 26707 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.66:38635
I20260806 22:49:42.280771 26890 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.66:38635 every 8 connection(s)
I20260806 22:49:42.281144 26707 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
I20260806 22:49:42.281397 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 26707
I20260806 22:49:42.281706 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.67:46713
--local_ip_for_outbound_sockets=127.6.40.67
--tserver_master_addrs=127.6.40.126:32939
--webserver_port=37237
--webserver_interface=127.6.40.67
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.67.keytab
--principal=kudu/127.6.40.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:42369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 22:49:42 dist-test-slave-zh2d krb5kdc[25586](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056582, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.66@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
I20260806 22:49:42.292048 26706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:42.281985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.66:33375 (local address 127.6.40.126:32939)
0806 22:49:42.282064 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:42.282066 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:42.282315 (+ 249us) server_negotiation.cc:407] Connection header received
0806 22:49:42.282814 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:42.282815 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:42.282834 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:42.282915 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:42.283983 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:42.284791 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:42.286371 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:42.286575 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:42.286658 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:42.288866 (+ 2208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:42.288867 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:42.288868 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:42.288875 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:42.290365 (+ 1490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:42.290914 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:42.290915 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:42.290915 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:42.290975 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:42.291325 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:42.291326 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:42.291326 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:42.291471 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:42.291546 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:42.291902 (+ 356us) server_negotiation.cc:299] Negotiation successful
0806 22:49:42.291981 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 22:49:42.292382 26893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:42.281942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:32939 (local address 127.6.40.66:33375)
0806 22:49:42.282141 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:42.282197 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:42.282713 (+ 516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:42.282992 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:42.282996 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:42.283586 (+ 590us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:49:42.283868 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:42.283875 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:42.284918 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:42.284920 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:42.286282 (+ 1362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:42.286286 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:42.286681 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:42.286682 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:42.286810 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:42.287352 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:49:42.287360 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:49:42.288746 (+ 1386us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:49:42.290465 (+ 1719us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:42.290474 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:42.290476 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:42.290833 (+ 357us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:42.291054 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:42.291056 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:42.291056 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:42.291216 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:42.291555 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:49:42.291563 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:49:42.291744 (+ 181us) client_negotiation.cc:769] Sending connection context
0806 22:49:42.291880 (+ 136us) client_negotiation.cc:240] Negotiation successful
0806 22:49:42.292084 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":107,"threads_started":1}
I20260806 22:49:42.292984 26891 heartbeater.cc:347] Connected to a master server at 127.6.40.126:32939
I20260806 22:49:42.293114 26891 heartbeater.cc:464] Registering TS with master...
I20260806 22:49:42.293314 26891 heartbeater.cc:511] Master 127.6.40.126:32939 requested a full tablet report, sending...
I20260806 22:49:42.293773 26430 ts_manager.cc:194] Registered new tserver with Master: 91e4eeee79304b0da16d36864b9fcc1f (127.6.40.66:38635)
I20260806 22:49:42.294354 26430 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.40.66@KRBTEST.COM'} at 127.6.40.66:33375
W20260806 22:49:42.369460 26894 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:49:42.369676 26894 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:49:42.369699 26894 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:49:42.370796 26894 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:49:42.370834 26894 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:49:42.370859 26894 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.67
I20260806 22:49:42.372241 26894 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:42369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.67.keytab
--principal=kudu/127.6.40.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.67:46713
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.40.67
--webserver_port=37237
--tserver_master_addrs=127.6.40.126:32939
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26894
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.67
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:49:42.372430 26894 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:49:42.372673 26894 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 22:49:42.378275 26894 server_base.cc:1034] running on GCE node
W20260806 22:49:42.378244 26903 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:42.378244 26907 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:42.378234 26900 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:42.378753 26894 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:49:42.379014 26894 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:49:42.380187 26894 hybrid_clock.cc:648] HybridClock initialized: now 1786056582380174 us; error 45 us; skew 500 ppm
Aug 06 22:49:42 dist-test-slave-zh2d krb5kdc[25586](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056582, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:49:42.382793 26894 init.cc:377] Logged in from keytab as kudu/127.6.40.67@KRBTEST.COM (short username kudu)
I20260806 22:49:42.383262 26894 webserver.cc:460] Webserver started at http://127.6.40.67:37237/ using document root <none> and password file <none>
I20260806 22:49:42.383502 26894 fs_manager.cc:359] Metadata directory not provided
I20260806 22:49:42.383594 26894 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:49:42.384965 26894 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:49:42.385576 26918 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:42.385746 26894 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:49:42.385850 26894 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-2/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-2/wal
uuid: "87fd367889cc4d379ad9e432f8572051"
format_stamp: "Formatted at 2026-08-06 22:49:41 on dist-test-slave-zh2d"
server_key: "cb95ff67629c4c6a77b7f7ae9809561c"
server_key_iv: "5dc2f637fd29ccca8c0c4edb225f396b"
server_key_version: "encryptionkey@0"
I20260806 22:49:42.386189 26894 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:49:42.399348 26894 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:49:42.401773 26894 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:49:42.401940 26894 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:49:42.402174 26894 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:49:42.402503 26894 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:49:42.402563 26894 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:42.402606 26894 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:49:42.402634 26894 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:42.408710 26894 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.67:46713
I20260806 22:49:42.408766 27082 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.67:46713 every 8 connection(s)
I20260806 22:49:42.409142 26894 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
Aug 06 22:49:42 dist-test-slave-zh2d krb5kdc[25586](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056582, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.67@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
I20260806 22:49:42.418409 26706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:42.409950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.67:50827 (local address 127.6.40.126:32939)
0806 22:49:42.410023 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:42.410026 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:42.410250 (+ 224us) server_negotiation.cc:407] Connection header received
0806 22:49:42.410742 (+ 492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:42.410743 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:42.410754 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:42.410808 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:42.411702 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:42.412260 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:42.413420 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:42.413685 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:42.413742 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:42.415849 (+ 2107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:42.415851 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:42.415851 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:42.415858 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:42.417137 (+ 1279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:42.417494 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:42.417495 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:42.417495 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:42.417565 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:42.417837 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:42.417838 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:42.417838 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:42.417944 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:42.417998 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:42.418279 (+ 281us) server_negotiation.cc:299] Negotiation successful
0806 22:49:42.418342 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 22:49:42.418609 27086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:42.409963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:32939 (local address 127.6.40.67:50827)
0806 22:49:42.410128 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:42.410165 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:42.410648 (+ 483us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:42.410858 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:42.410861 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:42.411425 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:49:42.411624 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:42.411629 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:42.412341 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:42.412342 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:42.413345 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:42.413348 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:42.413759 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:42.413760 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:42.413861 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:42.414397 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:49:42.414405 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:49:42.415763 (+ 1358us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:49:42.417205 (+ 1442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:42.417212 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:42.417214 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:42.417429 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:42.417673 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:42.417673 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:42.417674 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:42.417776 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:42.418004 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:49:42.418009 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:49:42.418165 (+ 156us) client_negotiation.cc:769] Sending connection context
0806 22:49:42.418250 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 22:49:42.418372 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":93,"threads_started":1}
I20260806 22:49:42.418751 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 26894
I20260806 22:49:42.419292 27083 heartbeater.cc:347] Connected to a master server at 127.6.40.126:32939
I20260806 22:49:42.419407 27083 heartbeater.cc:464] Registering TS with master...
I20260806 22:49:42.419607 27083 heartbeater.cc:511] Master 127.6.40.126:32939 requested a full tablet report, sending...
I20260806 22:49:42.420125 26430 ts_manager.cc:194] Registered new tserver with Master: 87fd367889cc4d379ad9e432f8572051 (127.6.40.67:46713)
I20260806 22:49:42.420773 26430 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.40.67@KRBTEST.COM'} at 127.6.40.67:50827
I20260806 22:49:42.425562 26706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:42.419144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51254 (local address 127.6.40.126:32939)
0806 22:49:42.419196 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:42.419198 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:42.419374 (+ 176us) server_negotiation.cc:407] Connection header received
0806 22:49:42.419403 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:42.419404 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:42.419416 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:42.419476 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:42.420206 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:42.421036 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:42.421980 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:42.422165 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:42.422219 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:42.423252 (+ 1033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:42.423254 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:42.423254 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:42.423259 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:42.424419 (+ 1160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:42.424728 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:42.424728 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:42.424728 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:42.424777 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:42.425095 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:42.425096 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:42.425096 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:42.425193 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:42.425326 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:42.425443 (+ 117us) server_negotiation.cc:299] Negotiation successful
0806 22:49:42.425499 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 22:49:42.426396 6305 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:49:42.429808 26706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:42.426774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51256 (local address 127.6.40.126:32939)
0806 22:49:42.426870 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:42.426872 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:42.426978 (+ 106us) server_negotiation.cc:407] Connection header received
0806 22:49:42.427108 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:42.427109 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:42.427152 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:42.427263 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:49:42.427477 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:42.427981 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:42.428880 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:42.429115 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:42.429216 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:42.429432 (+ 216us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:49:42.429542 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:49:42.429650 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:42.429696 (+ 46us) server_negotiation.cc:299] Negotiation successful
0806 22:49:42.429716 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 22:49:42.430130 26430 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:51256:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:49:42.431912 26430 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:49:42.434312 26584 tablet_service.cc:1467] Processing CreateTablet for tablet 51b0121ec03b47f1ab8f3ac3458d2a3d (DEFAULT_TABLE table=test-table [id=cdc5f8b9cd4d4c718734544c437bec8a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:49:42.434559 26584 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 51b0121ec03b47f1ab8f3ac3458d2a3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:49:42.436292 26539 tablet_bootstrap.cc:492] T 51b0121ec03b47f1ab8f3ac3458d2a3d P fd1b2e1656e24421813074d3a7a3f6bb: Bootstrap starting.
I20260806 22:49:42.436777 26539 tablet_bootstrap.cc:654] T 51b0121ec03b47f1ab8f3ac3458d2a3d P fd1b2e1656e24421813074d3a7a3f6bb: Neither blocks nor log segments found. Creating new log.
I20260806 22:49:42.437480 26539 tablet_bootstrap.cc:492] T 51b0121ec03b47f1ab8f3ac3458d2a3d P fd1b2e1656e24421813074d3a7a3f6bb: No bootstrap required, opened a new log
I20260806 22:49:42.437565 26539 ts_tablet_manager.cc:1397] T 51b0121ec03b47f1ab8f3ac3458d2a3d P fd1b2e1656e24421813074d3a7a3f6bb: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260806 22:49:42.437948 27094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:42.434187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39312 (local address 127.6.40.67:46713)
0806 22:49:42.434351 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:42.434355 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:42.434366 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:49:42.434405 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:42.434407 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:42.434423 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:42.434501 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:49:42.434729 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:42.435587 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:42.436709 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:42.437481 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:42.437575 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:42.437643 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:42.437812 (+ 169us) server_negotiation.cc:299] Negotiation successful
0806 22:49:42.437853 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":96,"threads_started":1}
I20260806 22:49:42.437948 26710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:42.433860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.66:38635 (local address 127.0.0.1:49006)
0806 22:49:42.433908 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:42.433920 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:42.433957 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:42.434381 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:42.434382 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:42.434384 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:49:42.434429 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:42.434430 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:42.435742 (+ 1312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:42.435743 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:42.436744 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:42.436747 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:42.437657 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:42.437658 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:42.437730 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:42.437733 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:49:42.437783 (+ 50us) client_negotiation.cc:240] Negotiation successful
0806 22:49:42.437836 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260806 22:49:42.437960 27090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:42.433903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.67:46713 (local address 127.0.0.1:39312)
0806 22:49:42.434011 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:42.434021 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:42.434068 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:42.434583 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:42.434584 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:42.434586 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:49:42.434649 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:42.434651 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:42.435690 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:42.435691 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:42.436611 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:42.436614 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:42.437608 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:42.437609 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:42.437684 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:42.437709 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 22:49:42.437786 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 22:49:42.437846 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92,"mutex_wait_us":13,"thread_start_us":69,"threads_started":1}
I20260806 22:49:42.437961 26539 raft_consensus.cc:348] T 51b0121ec03b47f1ab8f3ac3458d2a3d P fd1b2e1656e24421813074d3a7a3f6bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e4eeee79304b0da16d36864b9fcc1f" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 38635 } } peers { permanent_uuid: "87fd367889cc4d379ad9e432f8572051" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 46713 } } peers { permanent_uuid: "fd1b2e1656e24421813074d3a7a3f6bb" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 43385 } }
I20260806 22:49:42.438053 26539 raft_consensus.cc:374] T 51b0121ec03b47f1ab8f3ac3458d2a3d P fd1b2e1656e24421813074d3a7a3f6bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:49:42.438107 26539 raft_consensus.cc:729] T 51b0121ec03b47f1ab8f3ac3458d2a3d P fd1b2e1656e24421813074d3a7a3f6bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd1b2e1656e24421813074d3a7a3f6bb, State: Initialized, Role: FOLLOWER
I20260806 22:49:42.438163 26539 consensus_queue.cc:260] T 51b0121ec03b47f1ab8f3ac3458d2a3d P fd1b2e1656e24421813074d3a7a3f6bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e4eeee79304b0da16d36864b9fcc1f" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 38635 } } peers { permanent_uuid: "87fd367889cc4d379ad9e432f8572051" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 46713 } } peers { permanent_uuid: "fd1b2e1656e24421813074d3a7a3f6bb" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 43385 } }
I20260806 22:49:42.438294 26539 ts_tablet_manager.cc:1428] T 51b0121ec03b47f1ab8f3ac3458d2a3d P fd1b2e1656e24421813074d3a7a3f6bb: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260806 22:49:42.438364 27091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:42.433977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49006 (local address 127.6.40.66:38635)
0806 22:49:42.434187 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:42.434193 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:42.434204 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:49:42.434246 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:42.434248 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:42.434264 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:42.434509 (+ 245us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:49:42.434599 (+ 90us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:42.435609 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:42.436845 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:42.437563 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:42.438142 (+ 579us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:42.438217 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:42.438242 (+ 25us) server_negotiation.cc:299] Negotiation successful
0806 22:49:42.438282 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":114,"threads_started":1}
I20260806 22:49:42.438953 26982 tablet_service.cc:1467] Processing CreateTablet for tablet 51b0121ec03b47f1ab8f3ac3458d2a3d (DEFAULT_TABLE table=test-table [id=cdc5f8b9cd4d4c718734544c437bec8a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:49:42.439075 26798 tablet_service.cc:1467] Processing CreateTablet for tablet 51b0121ec03b47f1ab8f3ac3458d2a3d (DEFAULT_TABLE table=test-table [id=cdc5f8b9cd4d4c718734544c437bec8a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:49:42.439321 26982 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 51b0121ec03b47f1ab8f3ac3458d2a3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:49:42.439440 26798 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 51b0121ec03b47f1ab8f3ac3458d2a3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:49:42.442363 27098 tablet_bootstrap.cc:492] T 51b0121ec03b47f1ab8f3ac3458d2a3d P 87fd367889cc4d379ad9e432f8572051: Bootstrap starting.
I20260806 22:49:42.442526 27099 tablet_bootstrap.cc:492] T 51b0121ec03b47f1ab8f3ac3458d2a3d P 91e4eeee79304b0da16d36864b9fcc1f: Bootstrap starting.
I20260806 22:49:42.443126 27099 tablet_bootstrap.cc:654] T 51b0121ec03b47f1ab8f3ac3458d2a3d P 91e4eeee79304b0da16d36864b9fcc1f: Neither blocks nor log segments found. Creating new log.
I20260806 22:49:42.443136 27098 tablet_bootstrap.cc:654] T 51b0121ec03b47f1ab8f3ac3458d2a3d P 87fd367889cc4d379ad9e432f8572051: Neither blocks nor log segments found. Creating new log.
I20260806 22:49:42.443560 27099 log.cc:824] T 51b0121ec03b47f1ab8f3ac3458d2a3d P 91e4eeee79304b0da16d36864b9fcc1f: Log is configured to *not* fsync() on all Append() calls
I20260806 22:49:42.443704 27098 log.cc:824] T 51b0121ec03b47f1ab8f3ac3458d2a3d P 87fd367889cc4d379ad9e432f8572051: Log is configured to *not* fsync() on all Append() calls
I20260806 22:49:42.444291 27099 tablet_bootstrap.cc:492] T 51b0121ec03b47f1ab8f3ac3458d2a3d P 91e4eeee79304b0da16d36864b9fcc1f: No bootstrap required, opened a new log
I20260806 22:49:42.444348 27099 ts_tablet_manager.cc:1397] T 51b0121ec03b47f1ab8f3ac3458d2a3d P 91e4eeee79304b0da16d36864b9fcc1f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:49:42.444515 27098 tablet_bootstrap.cc:492] T 51b0121ec03b47f1ab8f3ac3458d2a3d P 87fd367889cc4d379ad9e432f8572051: No bootstrap required, opened a new log
I20260806 22:49:42.444619 27098 ts_tablet_manager.cc:1397] T 51b0121ec03b47f1ab8f3ac3458d2a3d P 87fd367889cc4d379ad9e432f8572051: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:49:42.446069 27099 raft_consensus.cc:348] T 51b0121ec03b47f1ab8f3ac3458d2a3d P 91e4eeee79304b0da16d36864b9fcc1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e4eeee79304b0da16d36864b9fcc1f" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 38635 } } peers { permanent_uuid: "87fd367889cc4d379ad9e432f8572051" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 46713 } } peers { permanent_uuid: "fd1b2e1656e24421813074d3a7a3f6bb" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 43385 } }
I20260806 22:49:42.446211 27099 raft_consensus.cc:374] T 51b0121ec03b47f1ab8f3ac3458d2a3d P 91e4eeee79304b0da16d36864b9fcc1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:49:42.446233 27099 raft_consensus.cc:729] T 51b0121ec03b47f1ab8f3ac3458d2a3d P 91e4eeee79304b0da16d36864b9fcc1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91e4eeee79304b0da16d36864b9fcc1f, State: Initialized, Role: FOLLOWER
I20260806 22:49:42.446345 27099 consensus_queue.cc:260] T 51b0121ec03b47f1ab8f3ac3458d2a3d P 91e4eeee79304b0da16d36864b9fcc1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e4eeee79304b0da16d36864b9fcc1f" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 38635 } } peers { permanent_uuid: "87fd367889cc4d379ad9e432f8572051" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 46713 } } peers { permanent_uuid: "fd1b2e1656e24421813074d3a7a3f6bb" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 43385 } }
I20260806 22:49:42.446303 27098 raft_consensus.cc:348] T 51b0121ec03b47f1ab8f3ac3458d2a3d P 87fd367889cc4d379ad9e432f8572051 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e4eeee79304b0da16d36864b9fcc1f" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 38635 } } peers { permanent_uuid: "87fd367889cc4d379ad9e432f8572051" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 46713 } } peers { permanent_uuid: "fd1b2e1656e24421813074d3a7a3f6bb" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 43385 } }
I20260806 22:49:42.446497 27098 raft_consensus.cc:374] T 51b0121ec03b47f1ab8f3ac3458d2a3d P 87fd367889cc4d379ad9e432f8572051 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:49:42.446540 27098 raft_consensus.cc:729] T 51b0121ec03b47f1ab8f3ac3458d2a3d P 87fd367889cc4d379ad9e432f8572051 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87fd367889cc4d379ad9e432f8572051, State: Initialized, Role: FOLLOWER
I20260806 22:49:42.446638 27099 ts_tablet_manager.cc:1428] T 51b0121ec03b47f1ab8f3ac3458d2a3d P 91e4eeee79304b0da16d36864b9fcc1f: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 22:49:42.446655 26891 heartbeater.cc:503] Master 127.6.40.126:32939 was elected leader, sending a full tablet report...
I20260806 22:49:42.446831 27098 consensus_queue.cc:260] T 51b0121ec03b47f1ab8f3ac3458d2a3d P 87fd367889cc4d379ad9e432f8572051 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e4eeee79304b0da16d36864b9fcc1f" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 38635 } } peers { permanent_uuid: "87fd367889cc4d379ad9e432f8572051" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 46713 } } peers { permanent_uuid: "fd1b2e1656e24421813074d3a7a3f6bb" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 43385 } }
I20260806 22:49:42.447170 27083 heartbeater.cc:503] Master 127.6.40.126:32939 was elected leader, sending a full tablet report...
I20260806 22:49:42.447186 27098 ts_tablet_manager.cc:1428] T 51b0121ec03b47f1ab8f3ac3458d2a3d P 87fd367889cc4d379ad9e432f8572051: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
W20260806 22:49:42.531878 26892 tablet.cc:2369] T 51b0121ec03b47f1ab8f3ac3458d2a3d P 91e4eeee79304b0da16d36864b9fcc1f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:49:42.576545 27102 raft_consensus.cc:482] T 51b0121ec03b47f1ab8f3ac3458d2a3d P 91e4eeee79304b0da16d36864b9fcc1f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:49:42.576676 27102 raft_consensus.cc:504] T 51b0121ec03b47f1ab8f3ac3458d2a3d P 91e4eeee79304b0da16d36864b9fcc1f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e4eeee79304b0da16d36864b9fcc1f" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 38635 } } peers { permanent_uuid: "87fd367889cc4d379ad9e432f8572051" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 46713 } } peers { permanent_uuid: "fd1b2e1656e24421813074d3a7a3f6bb" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 43385 } }
I20260806 22:49:42.577109 27102 leader_election.cc:288] T 51b0121ec03b47f1ab8f3ac3458d2a3d P 91e4eeee79304b0da16d36864b9fcc1f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 87fd367889cc4d379ad9e432f8572051 (127.6.40.67:46713), fd1b2e1656e24421813074d3a7a3f6bb (127.6.40.65:43385)
I20260806 22:49:42.580595 26893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:42.577320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.67:46713 (local address 127.6.40.66:38193)
0806 22:49:42.577407 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:42.577425 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:42.577506 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:42.577730 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:42.577732 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:42.577734 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:49:42.577817 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:42.577819 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:42.578849 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:42.578850 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:42.579659 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:42.579661 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:42.580316 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:42.580317 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:42.580409 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:42.580413 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:49:42.580468 (+ 55us) client_negotiation.cc:240] Negotiation successful
0806 22:49:42.580511 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260806 22:49:42.580646 27094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:42.577434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.66:38193 (local address 127.6.40.67:46713)
0806 22:49:42.577490 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:42.577493 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:42.577507 (+ 14us) server_negotiation.cc:407] Connection header received
0806 22:49:42.577626 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:42.577627 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:42.577642 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:42.577708 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:49:42.577904 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:42.578761 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:42.579761 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:42.580222 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:42.580286 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:42.580297 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:42.580499 (+ 202us) server_negotiation.cc:299] Negotiation successful
0806 22:49:42.580552 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 22:49:42.580755 27107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:42.577487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.65:43385 (local address 127.6.40.66:44593)
0806 22:49:42.577656 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:42.577665 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:42.577744 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:42.577959 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:42.577960 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:42.577962 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:49:42.578020 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:42.578021 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:42.578951 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:42.578952 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:42.579797 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:42.579801 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:42.580440 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:42.580441 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:42.580557 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:42.580562 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:49:42.580624 (+ 62us) client_negotiation.cc:240] Negotiation successful
0806 22:49:42.580665 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":106,"threads_started":1}
I20260806 22:49:42.580819 26711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:42.577542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.66:44593 (local address 127.6.40.65:43385)
0806 22:49:42.577606 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:42.577608 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:42.577754 (+ 146us) server_negotiation.cc:407] Connection header received
0806 22:49:42.577815 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:42.577816 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:42.577851 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:42.577945 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:49:42.578096 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:42.578875 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:42.579887 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:42.580337 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:42.580410 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:42.580527 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:42.580654 (+ 127us) server_negotiation.cc:299] Negotiation successful
0806 22:49:42.580707 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 22:49:42.581166 26628 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51b0121ec03b47f1ab8f3ac3458d2a3d" candidate_uuid: "91e4eeee79304b0da16d36864b9fcc1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd1b2e1656e24421813074d3a7a3f6bb" is_pre_election: true
I20260806 22:49:42.581207 27013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51b0121ec03b47f1ab8f3ac3458d2a3d" candidate_uuid: "91e4eeee79304b0da16d36864b9fcc1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87fd367889cc4d379ad9e432f8572051" is_pre_election: true
I20260806 22:49:42.581312 26628 raft_consensus.cc:2454] T 51b0121ec03b47f1ab8f3ac3458d2a3d P fd1b2e1656e24421813074d3a7a3f6bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91e4eeee79304b0da16d36864b9fcc1f in term 0.
I20260806 22:49:42.581370 27013 raft_consensus.cc:2454] T 51b0121ec03b47f1ab8f3ac3458d2a3d P 87fd367889cc4d379ad9e432f8572051 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91e4eeee79304b0da16d36864b9fcc1f in term 0.
I20260806 22:49:42.581634 26736 leader_election.cc:302] T 51b0121ec03b47f1ab8f3ac3458d2a3d P 91e4eeee79304b0da16d36864b9fcc1f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91e4eeee79304b0da16d36864b9fcc1f, fd1b2e1656e24421813074d3a7a3f6bb; no voters:
I20260806 22:49:42.581763 27102 raft_consensus.cc:2781] T 51b0121ec03b47f1ab8f3ac3458d2a3d P 91e4eeee79304b0da16d36864b9fcc1f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:49:42.581856 27102 raft_consensus.cc:482] T 51b0121ec03b47f1ab8f3ac3458d2a3d P 91e4eeee79304b0da16d36864b9fcc1f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:49:42.581890 27102 raft_consensus.cc:3037] T 51b0121ec03b47f1ab8f3ac3458d2a3d P 91e4eeee79304b0da16d36864b9fcc1f [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:49:42.582670 27102 raft_consensus.cc:504] T 51b0121ec03b47f1ab8f3ac3458d2a3d P 91e4eeee79304b0da16d36864b9fcc1f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e4eeee79304b0da16d36864b9fcc1f" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 38635 } } peers { permanent_uuid: "87fd367889cc4d379ad9e432f8572051" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 46713 } } peers { permanent_uuid: "fd1b2e1656e24421813074d3a7a3f6bb" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 43385 } }
I20260806 22:49:42.582823 27102 leader_election.cc:288] T 51b0121ec03b47f1ab8f3ac3458d2a3d P 91e4eeee79304b0da16d36864b9fcc1f [CANDIDATE]: Term 1 election: Requested vote from peers 87fd367889cc4d379ad9e432f8572051 (127.6.40.67:46713), fd1b2e1656e24421813074d3a7a3f6bb (127.6.40.65:43385)
I20260806 22:49:42.583062 27013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51b0121ec03b47f1ab8f3ac3458d2a3d" candidate_uuid: "91e4eeee79304b0da16d36864b9fcc1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87fd367889cc4d379ad9e432f8572051"
I20260806 22:49:42.583076 26628 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51b0121ec03b47f1ab8f3ac3458d2a3d" candidate_uuid: "91e4eeee79304b0da16d36864b9fcc1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd1b2e1656e24421813074d3a7a3f6bb"
I20260806 22:49:42.583158 26628 raft_consensus.cc:3037] T 51b0121ec03b47f1ab8f3ac3458d2a3d P fd1b2e1656e24421813074d3a7a3f6bb [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:49:42.583158 27013 raft_consensus.cc:3037] T 51b0121ec03b47f1ab8f3ac3458d2a3d P 87fd367889cc4d379ad9e432f8572051 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:49:42.584110 26628 raft_consensus.cc:2454] T 51b0121ec03b47f1ab8f3ac3458d2a3d P fd1b2e1656e24421813074d3a7a3f6bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91e4eeee79304b0da16d36864b9fcc1f in term 1.
I20260806 22:49:42.584110 27013 raft_consensus.cc:2454] T 51b0121ec03b47f1ab8f3ac3458d2a3d P 87fd367889cc4d379ad9e432f8572051 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91e4eeee79304b0da16d36864b9fcc1f in term 1.
I20260806 22:49:42.584282 26736 leader_election.cc:302] T 51b0121ec03b47f1ab8f3ac3458d2a3d P 91e4eeee79304b0da16d36864b9fcc1f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91e4eeee79304b0da16d36864b9fcc1f, fd1b2e1656e24421813074d3a7a3f6bb; no voters:
I20260806 22:49:42.584385 27102 raft_consensus.cc:2781] T 51b0121ec03b47f1ab8f3ac3458d2a3d P 91e4eeee79304b0da16d36864b9fcc1f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:49:42.584546 27102 raft_consensus.cc:686] T 51b0121ec03b47f1ab8f3ac3458d2a3d P 91e4eeee79304b0da16d36864b9fcc1f [term 1 LEADER]: Becoming Leader. State: Replica: 91e4eeee79304b0da16d36864b9fcc1f, State: Running, Role: LEADER
I20260806 22:49:42.584663 27102 consensus_queue.cc:237] T 51b0121ec03b47f1ab8f3ac3458d2a3d P 91e4eeee79304b0da16d36864b9fcc1f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e4eeee79304b0da16d36864b9fcc1f" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 38635 } } peers { permanent_uuid: "87fd367889cc4d379ad9e432f8572051" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 46713 } } peers { permanent_uuid: "fd1b2e1656e24421813074d3a7a3f6bb" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 43385 } }
I20260806 22:49:42.585578 26430 catalog_manager.cc:5495] T 51b0121ec03b47f1ab8f3ac3458d2a3d P 91e4eeee79304b0da16d36864b9fcc1f reported cstate change: term changed from 0 to 1, leader changed from <none> to 91e4eeee79304b0da16d36864b9fcc1f (127.6.40.66). New cstate: current_term: 1 leader_uuid: "91e4eeee79304b0da16d36864b9fcc1f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e4eeee79304b0da16d36864b9fcc1f" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 38635 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "87fd367889cc4d379ad9e432f8572051" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 46713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fd1b2e1656e24421813074d3a7a3f6bb" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 43385 } health_report { overall_health: UNKNOWN } } }
I20260806 22:49:42.619415 27091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:42.616885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49010 (local address 127.6.40.66:38635)
0806 22:49:42.616940 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:42.616942 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:42.616949 (+ 7us) server_negotiation.cc:407] Connection header received
0806 22:49:42.616987 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:42.616988 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:42.617010 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:42.617099 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:49:42.617215 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:42.617763 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:42.618593 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:42.618791 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:42.618869 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:42.619052 (+ 183us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:49:42.619149 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:49:42.619244 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:42.619273 (+ 29us) server_negotiation.cc:299] Negotiation successful
0806 22:49:42.619296 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 22:49:42.621440 27013 raft_consensus.cc:1261] T 51b0121ec03b47f1ab8f3ac3458d2a3d P 87fd367889cc4d379ad9e432f8572051 [term 1 FOLLOWER]: Refusing update from remote peer 91e4eeee79304b0da16d36864b9fcc1f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:49:42.621493 26628 raft_consensus.cc:1261] T 51b0121ec03b47f1ab8f3ac3458d2a3d P fd1b2e1656e24421813074d3a7a3f6bb [term 1 FOLLOWER]: Refusing update from remote peer 91e4eeee79304b0da16d36864b9fcc1f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:49:42.621870 27102 consensus_queue.cc:1035] T 51b0121ec03b47f1ab8f3ac3458d2a3d P 91e4eeee79304b0da16d36864b9fcc1f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd1b2e1656e24421813074d3a7a3f6bb" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 43385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:49:42.622066 27110 consensus_queue.cc:1035] T 51b0121ec03b47f1ab8f3ac3458d2a3d P 91e4eeee79304b0da16d36864b9fcc1f [LEADER]: Connected to new peer: Peer: permanent_uuid: "87fd367889cc4d379ad9e432f8572051" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 46713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:49:42.624826 27114 mvcc.cc:205] Tried to move back new op lower bound from 7315687762414309376 to 7315687762267521024. Current Snapshot: MvccSnapshot[applied={T|T < 7315687762414309376}]
I20260806 22:49:42.624951 27115 mvcc.cc:205] Tried to move back new op lower bound from 7315687762414309376 to 7315687762267521024. Current Snapshot: MvccSnapshot[applied={T|T < 7315687762414309376}]
I20260806 22:49:42.626888 26430 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:51256:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:49:42.626983 26430 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:51256:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:49:42.627676 26430 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b9e1eb63b32f4e9ea1c7f100f2d38d4c: Sending DeleteTablet for 3 replicas of tablet 51b0121ec03b47f1ab8f3ac3458d2a3d
I20260806 22:49:42.628041 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 26482
I20260806 22:49:42.628072 26982 tablet_service.cc:1514] Processing DeleteTablet for tablet 51b0121ec03b47f1ab8f3ac3458d2a3d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:49:42 UTC) from {username='kudu', principal='kudu/127.6.40.126@KRBTEST.COM'} at 127.0.0.1:39312
I20260806 22:49:42.628144 26798 tablet_service.cc:1514] Processing DeleteTablet for tablet 51b0121ec03b47f1ab8f3ac3458d2a3d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:49:42 UTC) from {username='kudu', principal='kudu/127.6.40.126@KRBTEST.COM'} at 127.0.0.1:49006
I20260806 22:49:42.634742 27127 tablet_replica.cc:330] T 51b0121ec03b47f1ab8f3ac3458d2a3d P 91e4eeee79304b0da16d36864b9fcc1f: stopping tablet replica
I20260806 22:49:42.634886 27127 raft_consensus.cc:2229] T 51b0121ec03b47f1ab8f3ac3458d2a3d P 91e4eeee79304b0da16d36864b9fcc1f [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:49:42.634990 27127 raft_consensus.cc:2258] T 51b0121ec03b47f1ab8f3ac3458d2a3d P 91e4eeee79304b0da16d36864b9fcc1f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:49:42.635146 27126 tablet_replica.cc:330] T 51b0121ec03b47f1ab8f3ac3458d2a3d P 87fd367889cc4d379ad9e432f8572051: stopping tablet replica
I20260806 22:49:42.635249 27126 raft_consensus.cc:2229] T 51b0121ec03b47f1ab8f3ac3458d2a3d P 87fd367889cc4d379ad9e432f8572051 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:49:42.635356 27126 raft_consensus.cc:2258] T 51b0121ec03b47f1ab8f3ac3458d2a3d P 87fd367889cc4d379ad9e432f8572051 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:49:42.636651 27126 ts_tablet_manager.cc:1905] T 51b0121ec03b47f1ab8f3ac3458d2a3d P 87fd367889cc4d379ad9e432f8572051: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:49:42.638844 27126 ts_tablet_manager.cc:1918] T 51b0121ec03b47f1ab8f3ac3458d2a3d P 87fd367889cc4d379ad9e432f8572051: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:49:42.638924 27126 log.cc:1196] T 51b0121ec03b47f1ab8f3ac3458d2a3d P 87fd367889cc4d379ad9e432f8572051: Deleting WAL directory at /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-2/wal/wals/51b0121ec03b47f1ab8f3ac3458d2a3d
I20260806 22:49:42.639055 27127 ts_tablet_manager.cc:1905] T 51b0121ec03b47f1ab8f3ac3458d2a3d P 91e4eeee79304b0da16d36864b9fcc1f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:49:42.639240 27126 ts_tablet_manager.cc:1939] T 51b0121ec03b47f1ab8f3ac3458d2a3d P 87fd367889cc4d379ad9e432f8572051: Deleting consensus metadata
I20260806 22:49:42.639729 26410 catalog_manager.cc:4841] TS 87fd367889cc4d379ad9e432f8572051 (127.6.40.67:46713): tablet 51b0121ec03b47f1ab8f3ac3458d2a3d (table test-table [id=cdc5f8b9cd4d4c718734544c437bec8a]) successfully deleted
I20260806 22:49:42.640668 27127 ts_tablet_manager.cc:1918] T 51b0121ec03b47f1ab8f3ac3458d2a3d P 91e4eeee79304b0da16d36864b9fcc1f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:49:42.640722 27127 log.cc:1196] T 51b0121ec03b47f1ab8f3ac3458d2a3d P 91e4eeee79304b0da16d36864b9fcc1f: Deleting WAL directory at /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-1/wal/wals/51b0121ec03b47f1ab8f3ac3458d2a3d
I20260806 22:49:42.641006 27127 ts_tablet_manager.cc:1939] T 51b0121ec03b47f1ab8f3ac3458d2a3d P 91e4eeee79304b0da16d36864b9fcc1f: Deleting consensus metadata
I20260806 22:49:42.643158 26413 catalog_manager.cc:4841] TS 91e4eeee79304b0da16d36864b9fcc1f (127.6.40.66:38635): tablet 51b0121ec03b47f1ab8f3ac3458d2a3d (table test-table [id=cdc5f8b9cd4d4c718734544c437bec8a]) successfully deleted
W20260806 22:49:42.654564 26410 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.40.65:43385 (error 108)
I20260806 22:49:42.654989 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 26707
I20260806 22:49:42.655174 27090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:42.655011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.65:43385 (local address 127.0.0.1:36578)
0806 22:49:42.655086 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:42.655135 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.65:43385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260806 22:49:42.659003 26410 catalog_manager.cc:4584] TS fd1b2e1656e24421813074d3a7a3f6bb (127.6.40.65:43385): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 51b0121ec03b47f1ab8f3ac3458d2a3d: Network error: Client connection negotiation failed: client connection to 127.6.40.65:43385: connect: Connection refused (error 111)
I20260806 22:49:42.663681 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 26894
I20260806 22:49:42.673835 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 26388
I20260806 22:49:42.687879 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.126:32939
--webserver_interface=127.6.40.126
--webserver_port=45679
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab
--principal=kudu/127.6.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:42369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.40.126:32939
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:49:42.772776 27129 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:49:42.773064 27129 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:49:42.773103 27129 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:49:42.774236 27129 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:49:42.774307 27129 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:49:42.774329 27129 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:49:42.774350 27129 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:49:42.774381 27129 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:49:42.775858 27129 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:42369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.40.126:32939
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab
--principal=kudu/127.6.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.126:32939
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.40.126
--webserver_port=45679
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27129
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:49:42.776144 27129 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:49:42.776382 27129 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:49:42.781424 27139 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:42.781427 27142 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:42.781463 27137 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:42.781581 27129 server_base.cc:1034] running on GCE node
I20260806 22:49:42.781952 27129 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:49:42.782256 27129 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:49:42.783425 27129 hybrid_clock.cc:648] HybridClock initialized: now 1786056582783413 us; error 30 us; skew 500 ppm
Aug 06 22:49:42 dist-test-slave-zh2d krb5kdc[25586](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056582, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:49:42.786019 27129 init.cc:377] Logged in from keytab as kudu/127.6.40.126@KRBTEST.COM (short username kudu)
I20260806 22:49:42.786479 27129 webserver.cc:460] Webserver started at http://127.6.40.126:45679/ using document root <none> and password file <none>
I20260806 22:49:42.786732 27129 fs_manager.cc:359] Metadata directory not provided
I20260806 22:49:42.786805 27129 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:49:42.788137 27129 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:49:42.788820 27152 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:42.789038 27129 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:49:42.789131 27129 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/master-0/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/master-0/wal
uuid: "b9e1eb63b32f4e9ea1c7f100f2d38d4c"
format_stamp: "Formatted at 2026-08-06 22:49:41 on dist-test-slave-zh2d"
server_key: "96801279d1363050246285ee624aa72c"
server_key_iv: "501ed5420d49cdc58d4f18edeeeed1a7"
server_key_version: "encryptionkey@0"
I20260806 22:49:42.789496 27129 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:49:42.827816 27129 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:49:42.829888 27129 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:49:42.830068 27129 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:49:42.834435 27129 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.126:32939
I20260806 22:49:42.834497 27236 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.126:32939 every 8 connection(s)
I20260806 22:49:42.834890 27129 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
I20260806 22:49:42.836493 27237 sys_catalog.cc:263] Verifying existing consensus state
I20260806 22:49:42.837157 27237 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9e1eb63b32f4e9ea1c7f100f2d38d4c: Bootstrap starting.
I20260806 22:49:42.844044 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 27129
I20260806 22:49:42.844411 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.65:43385
--local_ip_for_outbound_sockets=127.6.40.65
--tserver_master_addrs=127.6.40.126:32939
--webserver_port=38705
--webserver_interface=127.6.40.65
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.65.keytab
--principal=kudu/127.6.40.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:42369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:49:42.847149 27237 log.cc:824] T 00000000000000000000000000000000 P b9e1eb63b32f4e9ea1c7f100f2d38d4c: Log is configured to *not* fsync() on all Append() calls
I20260806 22:49:42.849699 27237 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9e1eb63b32f4e9ea1c7f100f2d38d4c: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 22:49:42.850013 27237 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9e1eb63b32f4e9ea1c7f100f2d38d4c: Bootstrap complete.
I20260806 22:49:42.852324 27237 raft_consensus.cc:348] T 00000000000000000000000000000000 P b9e1eb63b32f4e9ea1c7f100f2d38d4c [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9e1eb63b32f4e9ea1c7f100f2d38d4c" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 32939 } }
I20260806 22:49:42.852548 27237 raft_consensus.cc:729] T 00000000000000000000000000000000 P b9e1eb63b32f4e9ea1c7f100f2d38d4c [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9e1eb63b32f4e9ea1c7f100f2d38d4c, State: Initialized, Role: FOLLOWER
I20260806 22:49:42.852721 27237 consensus_queue.cc:260] T 00000000000000000000000000000000 P b9e1eb63b32f4e9ea1c7f100f2d38d4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9e1eb63b32f4e9ea1c7f100f2d38d4c" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 32939 } }
I20260806 22:49:42.852782 27237 raft_consensus.cc:388] T 00000000000000000000000000000000 P b9e1eb63b32f4e9ea1c7f100f2d38d4c [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:49:42.852859 27237 raft_consensus.cc:482] T 00000000000000000000000000000000 P b9e1eb63b32f4e9ea1c7f100f2d38d4c [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:49:42.852917 27237 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b9e1eb63b32f4e9ea1c7f100f2d38d4c [term 2 FOLLOWER]: Advancing to term 3
I20260806 22:49:42.853942 27237 raft_consensus.cc:504] T 00000000000000000000000000000000 P b9e1eb63b32f4e9ea1c7f100f2d38d4c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9e1eb63b32f4e9ea1c7f100f2d38d4c" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 32939 } }
I20260806 22:49:42.854030 27237 leader_election.cc:302] T 00000000000000000000000000000000 P b9e1eb63b32f4e9ea1c7f100f2d38d4c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b9e1eb63b32f4e9ea1c7f100f2d38d4c; no voters:
I20260806 22:49:42.854251 27237 leader_election.cc:288] T 00000000000000000000000000000000 P b9e1eb63b32f4e9ea1c7f100f2d38d4c [CANDIDATE]: Term 3 election: Requested vote from peers
I20260806 22:49:42.854286 27243 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b9e1eb63b32f4e9ea1c7f100f2d38d4c [term 3 FOLLOWER]: Leader election won for term 3
I20260806 22:49:42.854394 27243 raft_consensus.cc:686] T 00000000000000000000000000000000 P b9e1eb63b32f4e9ea1c7f100f2d38d4c [term 3 LEADER]: Becoming Leader. State: Replica: b9e1eb63b32f4e9ea1c7f100f2d38d4c, State: Running, Role: LEADER
I20260806 22:49:42.854537 27237 sys_catalog.cc:564] T 00000000000000000000000000000000 P b9e1eb63b32f4e9ea1c7f100f2d38d4c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:49:42.854562 27243 consensus_queue.cc:237] T 00000000000000000000000000000000 P b9e1eb63b32f4e9ea1c7f100f2d38d4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9e1eb63b32f4e9ea1c7f100f2d38d4c" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 32939 } }
I20260806 22:49:42.854969 27247 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9e1eb63b32f4e9ea1c7f100f2d38d4c [sys.catalog]: SysCatalogTable state changed. Reason: New leader b9e1eb63b32f4e9ea1c7f100f2d38d4c. Latest consensus state: current_term: 3 leader_uuid: "b9e1eb63b32f4e9ea1c7f100f2d38d4c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9e1eb63b32f4e9ea1c7f100f2d38d4c" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 32939 } } }
I20260806 22:49:42.855062 27247 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9e1eb63b32f4e9ea1c7f100f2d38d4c [sys.catalog]: This master's current role is: LEADER
I20260806 22:49:42.855638 27244 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9e1eb63b32f4e9ea1c7f100f2d38d4c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "b9e1eb63b32f4e9ea1c7f100f2d38d4c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9e1eb63b32f4e9ea1c7f100f2d38d4c" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 32939 } } }
I20260806 22:49:42.855791 27244 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9e1eb63b32f4e9ea1c7f100f2d38d4c [sys.catalog]: This master's current role is: LEADER
I20260806 22:49:42.856405 27266 catalog_manager.cc:1259] Loaded cluster ID: bf2abec72c424d718a8be5897a9eb258
I20260806 22:49:42.856455 27266 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b9e1eb63b32f4e9ea1c7f100f2d38d4c: loading cluster ID for follower catalog manager: success
W20260806 22:49:42.856753 27266 catalog_manager.cc:1576] T 00000000000000000000000000000000 P b9e1eb63b32f4e9ea1c7f100f2d38d4c: acquiring CA information for follower catalog manager: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
W20260806 22:49:42.856802 27266 catalog_manager.cc:873] Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO: failed to prepare follower catalog manager, will retry
I20260806 22:49:42.856858 27254 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:49:42.857246 27254 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:49:42.857316 27254 catalog_manager.cc:1259] Loaded cluster ID: bf2abec72c424d718a8be5897a9eb258
I20260806 22:49:42.857337 27254 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260806 22:49:42.857452 27254 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f2db3f8f9ad google::LogMessage::Fail() at ??:0
@ 0x7f2db3f9288a google::LogMessage::SendToLog() at ??:0
@ 0x7f2db3f8f4de google::LogMessage::Flush() at ??:0
@ 0x7f2db3f90a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f2dbbeb40f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f2dbbebd8b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f2db5e5d0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f2db5e52954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f2db6cb66db start_thread at ??:0
@ 0x7f2db2a8d61f clone at ??:0
W20260806 22:49:42.946589 27240 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:49:42.946790 27240 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:49:42.946815 27240 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:49:42.947968 27240 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:49:42.948009 27240 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:49:42.948037 27240 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.65
I20260806 22:49:42.949589 27240 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:42369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.65.keytab
--principal=kudu/127.6.40.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.65:43385
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.40.65
--webserver_port=38705
--tserver_master_addrs=127.6.40.126:32939
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27240
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.65
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:49:42.949847 27240 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:49:42.950089 27240 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:49:42.956427 27286 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:42.956432 27288 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:42.956626 27240 server_base.cc:1034] running on GCE node
W20260806 22:49:42.956437 27290 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:42.956941 27240 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:49:42.957206 27240 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:49:42.958370 27240 hybrid_clock.cc:648] HybridClock initialized: now 1786056582958354 us; error 33 us; skew 500 ppm
Aug 06 22:49:42 dist-test-slave-zh2d krb5kdc[25586](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056582, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:49:42.960937 27240 init.cc:377] Logged in from keytab as kudu/127.6.40.65@KRBTEST.COM (short username kudu)
I20260806 22:49:42.961433 27240 webserver.cc:460] Webserver started at http://127.6.40.65:38705/ using document root <none> and password file <none>
I20260806 22:49:42.961678 27240 fs_manager.cc:359] Metadata directory not provided
I20260806 22:49:42.961750 27240 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:49:42.963233 27240 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:49:42.964001 27298 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:42.964228 27240 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:49:42.964337 27240 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-0/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-0/wal
uuid: "fd1b2e1656e24421813074d3a7a3f6bb"
format_stamp: "Formatted at 2026-08-06 22:49:41 on dist-test-slave-zh2d"
server_key: "1debb8747148c3197bcef363e7b0d86f"
server_key_iv: "5852b2f46bb327f4ea1ba328b32abaad"
server_key_version: "encryptionkey@0"
I20260806 22:49:42.964854 27240 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:49:43.002974 27240 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:49:43.006026 27240 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:49:43.006243 27240 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:49:43.006552 27240 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:49:43.007109 27306 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 22:49:43.008701 27240 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 22:49:43.008777 27240 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260806 22:49:43.008826 27240 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 22:49:43.009923 27240 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 22:49:43.009980 27240 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260806 22:49:43.010102 27306 tablet_bootstrap.cc:492] T 51b0121ec03b47f1ab8f3ac3458d2a3d P fd1b2e1656e24421813074d3a7a3f6bb: Bootstrap starting.
I20260806 22:49:43.018980 27240 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.65:43385
I20260806 22:49:43.019071 27449 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.65:43385 every 8 connection(s)
I20260806 22:49:43.019599 27240 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
I20260806 22:49:43.021684 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 27240
I20260806 22:49:43.021992 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.66:38635
--local_ip_for_outbound_sockets=127.6.40.66
--tserver_master_addrs=127.6.40.126:32939
--webserver_port=44203
--webserver_interface=127.6.40.66
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.66.keytab
--principal=kudu/127.6.40.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:42369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:49:43.022166 27306 log.cc:824] T 51b0121ec03b47f1ab8f3ac3458d2a3d P fd1b2e1656e24421813074d3a7a3f6bb: Log is configured to *not* fsync() on all Append() calls
I20260806 22:49:43.024883 27306 tablet_bootstrap.cc:492] T 51b0121ec03b47f1ab8f3ac3458d2a3d P fd1b2e1656e24421813074d3a7a3f6bb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 22:49:43.025223 27306 tablet_bootstrap.cc:492] T 51b0121ec03b47f1ab8f3ac3458d2a3d P fd1b2e1656e24421813074d3a7a3f6bb: Bootstrap complete.
I20260806 22:49:43.026357 27306 ts_tablet_manager.cc:1397] T 51b0121ec03b47f1ab8f3ac3458d2a3d P fd1b2e1656e24421813074d3a7a3f6bb: Time spent bootstrapping tablet: real 0.016s user 0.005s sys 0.009s
I20260806 22:49:43.028396 27306 raft_consensus.cc:348] T 51b0121ec03b47f1ab8f3ac3458d2a3d P fd1b2e1656e24421813074d3a7a3f6bb [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e4eeee79304b0da16d36864b9fcc1f" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 38635 } } peers { permanent_uuid: "87fd367889cc4d379ad9e432f8572051" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 46713 } } peers { permanent_uuid: "fd1b2e1656e24421813074d3a7a3f6bb" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 43385 } }
I20260806 22:49:43.028820 27306 raft_consensus.cc:729] T 51b0121ec03b47f1ab8f3ac3458d2a3d P fd1b2e1656e24421813074d3a7a3f6bb [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd1b2e1656e24421813074d3a7a3f6bb, State: Initialized, Role: FOLLOWER
I20260806 22:49:43.029048 27306 consensus_queue.cc:260] T 51b0121ec03b47f1ab8f3ac3458d2a3d P fd1b2e1656e24421813074d3a7a3f6bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e4eeee79304b0da16d36864b9fcc1f" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 38635 } } peers { permanent_uuid: "87fd367889cc4d379ad9e432f8572051" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 46713 } } peers { permanent_uuid: "fd1b2e1656e24421813074d3a7a3f6bb" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 43385 } }
I20260806 22:49:43.029414 27306 ts_tablet_manager.cc:1428] T 51b0121ec03b47f1ab8f3ac3458d2a3d P fd1b2e1656e24421813074d3a7a3f6bb: Time spent starting tablet: real 0.003s user 0.001s sys 0.003s
W20260806 22:49:43.063166 27452 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:49:43.020460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:32939 (local address 127.6.40.65:56361)
0806 22:49:43.020690 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:43.020755 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:43.021692 (+ 937us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:43.062912 (+ 41220us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:32939: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":114,"threads_started":1}
W20260806 22:49:43.063463 27450 heartbeater.cc:644] Failed to heartbeat to 127.6.40.126:32939 (0 consecutive failures): Network error: Failed to ping master at 127.6.40.126:32939: Client connection negotiation failed: client connection to 127.6.40.126:32939: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 22:49:43.063958 27452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:43.063785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:32939 (local address 127.6.40.65:48331)
0806 22:49:43.063867 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:43.063925 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:32939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260806 22:49:43.064381 27452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:43.064267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:32939 (local address 127.6.40.65:42165)
0806 22:49:43.064321 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:43.064347 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:32939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260806 22:49:43.064549 27450 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 22:49:43.111711 27456 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:49:43.111920 27456 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:49:43.111944 27456 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:49:43.113128 27456 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:49:43.113169 27456 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:49:43.113195 27456 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.66
I20260806 22:49:43.114626 27456 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:42369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.66.keytab
--principal=kudu/127.6.40.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.66:38635
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.40.66
--webserver_port=44203
--tserver_master_addrs=127.6.40.126:32939
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27456
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.66
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:49:43.114862 27456 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:49:43.115116 27456 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:49:43.120501 27466 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:43.120492 27470 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:43.120492 27467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:43.120900 27456 server_base.cc:1034] running on GCE node
I20260806 22:49:43.121073 27456 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:49:43.121279 27456 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:49:43.122416 27456 hybrid_clock.cc:648] HybridClock initialized: now 1786056583122404 us; error 30 us; skew 500 ppm
Aug 06 22:49:43 dist-test-slave-zh2d krb5kdc[25586](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056583, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:49:43.124943 27456 init.cc:377] Logged in from keytab as kudu/127.6.40.66@KRBTEST.COM (short username kudu)
I20260806 22:49:43.125393 27456 webserver.cc:460] Webserver started at http://127.6.40.66:44203/ using document root <none> and password file <none>
I20260806 22:49:43.125586 27456 fs_manager.cc:359] Metadata directory not provided
I20260806 22:49:43.125628 27456 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:49:43.126906 27456 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:49:43.127521 27478 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:43.127753 27456 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:49:43.127830 27456 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-1/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-1/wal
uuid: "91e4eeee79304b0da16d36864b9fcc1f"
format_stamp: "Formatted at 2026-08-06 22:49:41 on dist-test-slave-zh2d"
server_key: "9319cfeecf264e9c4274d57891c2e5a9"
server_key_iv: "23407928f59f72f10f6975735c620a85"
server_key_version: "encryptionkey@0"
I20260806 22:49:43.128149 27456 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:49:43.148284 27456 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:49:43.150321 27456 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:49:43.150442 27456 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:49:43.150705 27456 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:49:43.150986 27456 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:49:43.151018 27456 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:43.151041 27456 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:49:43.151052 27456 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:43.157390 27456 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.66:38635
I20260806 22:49:43.157444 27637 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.66:38635 every 8 connection(s)
I20260806 22:49:43.157773 27456 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
I20260806 22:49:43.158798 27641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:43.158422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:32939 (local address 127.6.40.66:46365)
0806 22:49:43.158578 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:43.158651 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:32939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":71,"threads_started":1}
W20260806 22:49:43.159251 27638 heartbeater.cc:644] Failed to heartbeat to 127.6.40.126:32939 (0 consecutive failures): Network error: Failed to ping master at 127.6.40.126:32939: Client connection negotiation failed: client connection to 127.6.40.126:32939: connect: Connection refused (error 111)
I20260806 22:49:43.159524 27641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:43.159435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:32939 (local address 127.6.40.66:51249)
0806 22:49:43.159478 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:43.159506 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:32939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260806 22:49:43.159754 27641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:43.159705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:32939 (local address 127.6.40.66:43805)
0806 22:49:43.159725 (+ 20us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:43.159742 (+ 17us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:32939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12}
W20260806 22:49:43.159855 27638 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 22:49:43.160758 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 27456
I20260806 22:49:43.161075 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.67:46713
--local_ip_for_outbound_sockets=127.6.40.67
--tserver_master_addrs=127.6.40.126:32939
--webserver_port=37237
--webserver_interface=127.6.40.67
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.67.keytab
--principal=kudu/127.6.40.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:42369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:49:43.245635 27642 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:49:43.245882 27642 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:49:43.245920 27642 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:49:43.247059 27642 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:49:43.247129 27642 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:49:43.247174 27642 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.67
I20260806 22:49:43.248620 27642 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:42369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.67.keytab
--principal=kudu/127.6.40.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.67:46713
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.40.67
--webserver_port=37237
--tserver_master_addrs=127.6.40.126:32939
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27642
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.67
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:49:43.248909 27642 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:49:43.249147 27642 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 22:49:43.254566 27642 server_base.cc:1034] running on GCE node
W20260806 22:49:43.254670 27655 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:43.254593 27648 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:43.254593 27649 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:43.255012 27642 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:49:43.255265 27642 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:49:43.256456 27642 hybrid_clock.cc:648] HybridClock initialized: now 1786056583256445 us; error 44 us; skew 500 ppm
Aug 06 22:49:43 dist-test-slave-zh2d krb5kdc[25586](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056583, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:49:43.259073 27642 init.cc:377] Logged in from keytab as kudu/127.6.40.67@KRBTEST.COM (short username kudu)
I20260806 22:49:43.259582 27642 webserver.cc:460] Webserver started at http://127.6.40.67:37237/ using document root <none> and password file <none>
I20260806 22:49:43.259811 27642 fs_manager.cc:359] Metadata directory not provided
I20260806 22:49:43.259882 27642 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:49:43.261353 27642 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260806 22:49:43.261976 27664 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:43.262146 27642 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:43.262251 27642 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-2/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-2/wal
uuid: "87fd367889cc4d379ad9e432f8572051"
format_stamp: "Formatted at 2026-08-06 22:49:41 on dist-test-slave-zh2d"
server_key: "cb95ff67629c4c6a77b7f7ae9809561c"
server_key_iv: "5dc2f637fd29ccca8c0c4edb225f396b"
server_key_version: "encryptionkey@0"
I20260806 22:49:43.262627 27642 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:49:43.281973 27642 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:49:43.284504 27642 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:49:43.284687 27642 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:49:43.284987 27642 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:49:43.285316 27642 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:49:43.285372 27642 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:43.285413 27642 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:49:43.285466 27642 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:43.291638 27642 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.67:46713
I20260806 22:49:43.291709 27816 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.67:46713 every 8 connection(s)
I20260806 22:49:43.292032 27642 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
I20260806 22:49:43.293226 27820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:43.292820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:32939 (local address 127.6.40.67:45343)
0806 22:49:43.292989 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:43.293075 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:32939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":75,"threads_started":1}
W20260806 22:49:43.293633 27817 heartbeater.cc:644] Failed to heartbeat to 127.6.40.126:32939 (0 consecutive failures): Network error: Failed to ping master at 127.6.40.126:32939: Client connection negotiation failed: client connection to 127.6.40.126:32939: connect: Connection refused (error 111)
I20260806 22:49:43.293936 27820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:43.293834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:32939 (local address 127.6.40.67:42535)
0806 22:49:43.293886 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:43.293916 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:32939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260806 22:49:43.294231 27820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:43.294143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:32939 (local address 127.6.40.67:40777)
0806 22:49:43.294186 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:43.294210 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:32939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260806 22:49:43.294332 27817 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 22:49:43.297230 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 27642
I20260806 22:49:44.065318 27824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:44.065043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:32939 (local address 127.6.40.65:47775)
0806 22:49:44.065179 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:44.065240 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:32939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":64,"threads_started":1}
I20260806 22:49:44.160470 27826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:44.160204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:32939 (local address 127.6.40.66:35411)
0806 22:49:44.160353 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:44.160393 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:32939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":65,"threads_started":1}
I20260806 22:49:44.294857 27827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:44.294624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:32939 (local address 127.6.40.67:54327)
0806 22:49:44.294759 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:44.294799 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:32939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":59,"threads_started":1}
I20260806 22:49:44.818773 27828 raft_consensus.cc:482] T 51b0121ec03b47f1ab8f3ac3458d2a3d P fd1b2e1656e24421813074d3a7a3f6bb [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:49:44.818907 27828 raft_consensus.cc:504] T 51b0121ec03b47f1ab8f3ac3458d2a3d P fd1b2e1656e24421813074d3a7a3f6bb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e4eeee79304b0da16d36864b9fcc1f" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 38635 } } peers { permanent_uuid: "87fd367889cc4d379ad9e432f8572051" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 46713 } } peers { permanent_uuid: "fd1b2e1656e24421813074d3a7a3f6bb" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 43385 } }
I20260806 22:49:44.819221 27828 leader_election.cc:288] T 51b0121ec03b47f1ab8f3ac3458d2a3d P fd1b2e1656e24421813074d3a7a3f6bb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91e4eeee79304b0da16d36864b9fcc1f (127.6.40.66:38635), 87fd367889cc4d379ad9e432f8572051 (127.6.40.67:46713)
Aug 06 22:49:44 dist-test-slave-zh2d krb5kdc[25586](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056582, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.65@KRBTEST.COM for kudu/127.6.40.66@KRBTEST.COM
Aug 06 22:49:44 dist-test-slave-zh2d krb5kdc[25586](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056582, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.65@KRBTEST.COM for kudu/127.6.40.67@KRBTEST.COM
I20260806 22:49:44.836653 27833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:44.819425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.66:38635 (local address 127.6.40.65:54639)
0806 22:49:44.819599 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:44.819612 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:44.819686 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:44.820758 (+ 1072us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:44.820762 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:44.821320 (+ 558us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:49:44.821604 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:44.821612 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:44.823318 (+ 1706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:44.823319 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:44.824496 (+ 1177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:44.824500 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:44.825354 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:44.825355 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:44.825533 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:44.830980 (+ 5447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:49:44.830992 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:49:44.832965 (+ 1973us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:49:44.834965 (+ 2000us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:44.834974 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:44.834976 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:44.835257 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:44.835552 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:44.835553 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:44.835553 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:44.835690 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:44.836112 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:49:44.836121 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:49:44.836256 (+ 135us) client_negotiation.cc:769] Sending connection context
0806 22:49:44.836361 (+ 105us) client_negotiation.cc:240] Negotiation successful
0806 22:49:44.836538 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"mutex_wait_us":2,"thread_start_us":74,"threads_started":1}
I20260806 22:49:44.836674 27834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:44.819588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.65:54639 (local address 127.6.40.66:38635)
0806 22:49:44.819782 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:44.819795 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:44.819822 (+ 27us) server_negotiation.cc:407] Connection header received
0806 22:49:44.820427 (+ 605us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:44.820431 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:44.820494 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:44.820663 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:44.821764 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:44.823208 (+ 1444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:44.824612 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:44.825243 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:44.825356 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:44.833133 (+ 7777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:44.833138 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:44.833138 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:44.833151 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:44.834864 (+ 1713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:44.835362 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:44.835366 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:44.835366 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:44.835455 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:44.835759 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:44.835761 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:44.835761 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:44.836022 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:44.836119 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:44.836435 (+ 316us) server_negotiation.cc:299] Negotiation successful
0806 22:49:44.836576 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":67,"threads_started":1}
I20260806 22:49:44.837368 27831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:44.819425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.67:46713 (local address 127.6.40.65:48915)
0806 22:49:44.819606 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:44.819617 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:44.819686 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:44.820758 (+ 1072us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:44.820762 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:44.821321 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:49:44.821604 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:44.821612 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:44.823673 (+ 2061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:44.823674 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:44.824643 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:44.824646 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:44.825403 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:44.825403 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:44.825533 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:44.830980 (+ 5447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:49:44.830992 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:49:44.833295 (+ 2303us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:49:44.835887 (+ 2592us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:44.835888 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:44.835888 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:44.836081 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:44.836376 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:44.836377 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:44.836378 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:44.836452 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:44.837014 (+ 562us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:49:44.837015 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:49:44.837068 (+ 53us) client_negotiation.cc:769] Sending connection context
0806 22:49:44.837153 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 22:49:44.837259 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"mutex_wait_us":6,"thread_start_us":69,"threads_started":1}
I20260806 22:49:44.837536 27836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:44.819607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.65:48915 (local address 127.6.40.67:46713)
0806 22:49:44.819782 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:44.819795 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:44.819822 (+ 27us) server_negotiation.cc:407] Connection header received
0806 22:49:44.820427 (+ 605us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:44.820431 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:44.820494 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:44.820663 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:44.821803 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:44.823537 (+ 1734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:44.824734 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:44.825326 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:44.825422 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:44.833419 (+ 7997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:44.833423 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:44.833423 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:44.833453 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:44.835773 (+ 2320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:44.836188 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:44.836191 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:44.836192 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:44.836299 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:44.836522 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:44.836523 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:44.836524 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:44.836873 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:44.836976 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:44.837237 (+ 261us) server_negotiation.cc:299] Negotiation successful
0806 22:49:44.837359 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":64,"threads_started":1}
W20260806 22:49:44.837646 27302 leader_election.cc:341] T 51b0121ec03b47f1ab8f3ac3458d2a3d P fd1b2e1656e24421813074d3a7a3f6bb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 91e4eeee79304b0da16d36864b9fcc1f (127.6.40.66:38635): Not found: Tablet not found: 51b0121ec03b47f1ab8f3ac3458d2a3d
W20260806 22:49:44.838306 27299 leader_election.cc:341] T 51b0121ec03b47f1ab8f3ac3458d2a3d P fd1b2e1656e24421813074d3a7a3f6bb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 87fd367889cc4d379ad9e432f8572051 (127.6.40.67:46713): Not found: Tablet not found: 51b0121ec03b47f1ab8f3ac3458d2a3d
I20260806 22:49:44.838366 27299 leader_election.cc:302] T 51b0121ec03b47f1ab8f3ac3458d2a3d P fd1b2e1656e24421813074d3a7a3f6bb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fd1b2e1656e24421813074d3a7a3f6bb; no voters: 87fd367889cc4d379ad9e432f8572051, 91e4eeee79304b0da16d36864b9fcc1f
I20260806 22:49:44.838480 27828 raft_consensus.cc:2726] T 51b0121ec03b47f1ab8f3ac3458d2a3d P fd1b2e1656e24421813074d3a7a3f6bb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 22:49:45.066025 27831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:45.065848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:32939 (local address 127.6.40.65:50125)
0806 22:49:45.065906 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:45.065978 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:32939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260806 22:49:45.161283 27837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:45.161058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:32939 (local address 127.6.40.66:47773)
0806 22:49:45.161187 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:45.161234 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:32939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":66,"threads_started":1}
I20260806 22:49:45.295665 27838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:45.295414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:32939 (local address 127.6.40.67:53839)
0806 22:49:45.295562 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:45.295616 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:32939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":70,"threads_started":1}
I20260806 22:49:46.066876 27840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:46.066604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:32939 (local address 127.6.40.65:50721)
0806 22:49:46.066755 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:46.066799 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:32939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":78,"threads_started":1}
I20260806 22:49:46.161975 27841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:46.161719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:32939 (local address 127.6.40.66:60733)
0806 22:49:46.161880 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:46.161925 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:32939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":70,"threads_started":1}
I20260806 22:49:46.296442 27842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:46.296201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:32939 (local address 127.6.40.67:37015)
0806 22:49:46.296344 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:46.296385 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:32939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":68,"threads_started":1}
I20260806 22:49:47.057195 27843 raft_consensus.cc:482] T 51b0121ec03b47f1ab8f3ac3458d2a3d P fd1b2e1656e24421813074d3a7a3f6bb [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:49:47.057305 27843 raft_consensus.cc:504] T 51b0121ec03b47f1ab8f3ac3458d2a3d P fd1b2e1656e24421813074d3a7a3f6bb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e4eeee79304b0da16d36864b9fcc1f" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 38635 } } peers { permanent_uuid: "87fd367889cc4d379ad9e432f8572051" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 46713 } } peers { permanent_uuid: "fd1b2e1656e24421813074d3a7a3f6bb" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 43385 } }
I20260806 22:49:47.057538 27843 leader_election.cc:288] T 51b0121ec03b47f1ab8f3ac3458d2a3d P fd1b2e1656e24421813074d3a7a3f6bb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91e4eeee79304b0da16d36864b9fcc1f (127.6.40.66:38635), 87fd367889cc4d379ad9e432f8572051 (127.6.40.67:46713)
W20260806 22:49:47.057983 27302 leader_election.cc:341] T 51b0121ec03b47f1ab8f3ac3458d2a3d P fd1b2e1656e24421813074d3a7a3f6bb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 91e4eeee79304b0da16d36864b9fcc1f (127.6.40.66:38635): Not found: Tablet not found: 51b0121ec03b47f1ab8f3ac3458d2a3d
W20260806 22:49:47.058054 27299 leader_election.cc:341] T 51b0121ec03b47f1ab8f3ac3458d2a3d P fd1b2e1656e24421813074d3a7a3f6bb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 87fd367889cc4d379ad9e432f8572051 (127.6.40.67:46713): Not found: Tablet not found: 51b0121ec03b47f1ab8f3ac3458d2a3d
I20260806 22:49:47.058082 27302 leader_election.cc:302] T 51b0121ec03b47f1ab8f3ac3458d2a3d P fd1b2e1656e24421813074d3a7a3f6bb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fd1b2e1656e24421813074d3a7a3f6bb; no voters: 87fd367889cc4d379ad9e432f8572051, 91e4eeee79304b0da16d36864b9fcc1f
I20260806 22:49:47.058156 27843 raft_consensus.cc:2726] T 51b0121ec03b47f1ab8f3ac3458d2a3d P fd1b2e1656e24421813074d3a7a3f6bb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 22:49:47.067531 27844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:47.067338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:32939 (local address 127.6.40.65:37529)
0806 22:49:47.067454 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:47.067490 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:32939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":62,"threads_started":1}
I20260806 22:49:47.162781 27846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:47.162500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:32939 (local address 127.6.40.66:55005)
0806 22:49:47.162665 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:47.162719 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:32939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":82,"threads_started":1}
I20260806 22:49:47.297305 27847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:47.297052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:32939 (local address 127.6.40.67:41105)
0806 22:49:47.297198 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:47.297240 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:32939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":79,"threads_started":1}
I20260806 22:49:48.068276 27848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:48.068046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:32939 (local address 127.6.40.65:39473)
0806 22:49:48.068183 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:48.068223 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:32939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":63,"threads_started":1}
I20260806 22:49:48.163496 27849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:48.163296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:32939 (local address 127.6.40.66:45427)
0806 22:49:48.163418 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:48.163453 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:32939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":66,"threads_started":1}
I20260806 22:49:48.298134 27850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:48.297832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:32939 (local address 127.6.40.67:34507)
0806 22:49:48.297998 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:48.298070 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:32939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":81,"threads_started":1}
I20260806 22:49:49.069175 27851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:49.068934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:32939 (local address 127.6.40.65:43067)
0806 22:49:49.069087 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:49.069124 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:32939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":86,"threads_started":1}
I20260806 22:49:49.164196 27852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:49.163953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:32939 (local address 127.6.40.66:59651)
0806 22:49:49.164107 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:49.164148 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:32939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":81,"threads_started":1}
I20260806 22:49:49.298925 27854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:49.298687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:32939 (local address 127.6.40.67:38731)
0806 22:49:49.298844 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:49.298881 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:32939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":74,"threads_started":1}
I20260806 22:49:49.743623 27855 raft_consensus.cc:482] T 51b0121ec03b47f1ab8f3ac3458d2a3d P fd1b2e1656e24421813074d3a7a3f6bb [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:49:49.743729 27855 raft_consensus.cc:504] T 51b0121ec03b47f1ab8f3ac3458d2a3d P fd1b2e1656e24421813074d3a7a3f6bb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e4eeee79304b0da16d36864b9fcc1f" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 38635 } } peers { permanent_uuid: "87fd367889cc4d379ad9e432f8572051" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 46713 } } peers { permanent_uuid: "fd1b2e1656e24421813074d3a7a3f6bb" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 43385 } }
I20260806 22:49:49.743979 27855 leader_election.cc:288] T 51b0121ec03b47f1ab8f3ac3458d2a3d P fd1b2e1656e24421813074d3a7a3f6bb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91e4eeee79304b0da16d36864b9fcc1f (127.6.40.66:38635), 87fd367889cc4d379ad9e432f8572051 (127.6.40.67:46713)
W20260806 22:49:49.744443 27299 leader_election.cc:341] T 51b0121ec03b47f1ab8f3ac3458d2a3d P fd1b2e1656e24421813074d3a7a3f6bb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 87fd367889cc4d379ad9e432f8572051 (127.6.40.67:46713): Not found: Tablet not found: 51b0121ec03b47f1ab8f3ac3458d2a3d
W20260806 22:49:49.744565 27302 leader_election.cc:341] T 51b0121ec03b47f1ab8f3ac3458d2a3d P fd1b2e1656e24421813074d3a7a3f6bb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 91e4eeee79304b0da16d36864b9fcc1f (127.6.40.66:38635): Not found: Tablet not found: 51b0121ec03b47f1ab8f3ac3458d2a3d
I20260806 22:49:49.744622 27302 leader_election.cc:302] T 51b0121ec03b47f1ab8f3ac3458d2a3d P fd1b2e1656e24421813074d3a7a3f6bb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fd1b2e1656e24421813074d3a7a3f6bb; no voters: 87fd367889cc4d379ad9e432f8572051, 91e4eeee79304b0da16d36864b9fcc1f
I20260806 22:49:49.744725 27855 raft_consensus.cc:2726] T 51b0121ec03b47f1ab8f3ac3458d2a3d P fd1b2e1656e24421813074d3a7a3f6bb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 22:49:50.069978 27856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:50.069666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:32939 (local address 127.6.40.65:49149)
0806 22:49:50.069849 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:50.069899 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:32939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":109,"threads_started":1}
I20260806 22:49:50.165048 27858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:50.164703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:32939 (local address 127.6.40.66:59421)
0806 22:49:50.164926 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:50.164979 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:32939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":105,"threads_started":1}
I20260806 22:49:50.299770 27859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:50.299488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:32939 (local address 127.6.40.67:38177)
0806 22:49:50.299673 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:50.299722 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:32939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":100,"threads_started":1}
I20260806 22:49:51.070760 27861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:51.070486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:32939 (local address 127.6.40.65:49399)
0806 22:49:51.070658 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:51.070703 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:32939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":80,"threads_started":1}
I20260806 22:49:51.165742 27862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:51.165493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:32939 (local address 127.6.40.66:37519)
0806 22:49:51.165648 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:51.165694 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:32939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":71,"threads_started":1}
I20260806 22:49:51.300590 27863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:51.300354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:32939 (local address 127.6.40.67:55931)
0806 22:49:51.300496 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:51.300537 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:32939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":72,"threads_started":1}
I20260806 22:49:52.071666 27865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:52.071418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:32939 (local address 127.6.40.65:35263)
0806 22:49:52.071576 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:52.071616 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:32939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":87,"threads_started":1}
I20260806 22:49:52.166483 27866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:52.166197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:32939 (local address 127.6.40.66:46945)
0806 22:49:52.166389 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:52.166433 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:32939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":93,"threads_started":1}
I20260806 22:49:52.301366 27867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:52.301109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:32939 (local address 127.6.40.67:45485)
0806 22:49:52.301265 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:52.301314 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:32939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":82,"threads_started":1}
I20260806 22:49:52.442770 27869 raft_consensus.cc:482] T 51b0121ec03b47f1ab8f3ac3458d2a3d P fd1b2e1656e24421813074d3a7a3f6bb [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:49:52.442876 27869 raft_consensus.cc:504] T 51b0121ec03b47f1ab8f3ac3458d2a3d P fd1b2e1656e24421813074d3a7a3f6bb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e4eeee79304b0da16d36864b9fcc1f" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 38635 } } peers { permanent_uuid: "87fd367889cc4d379ad9e432f8572051" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 46713 } } peers { permanent_uuid: "fd1b2e1656e24421813074d3a7a3f6bb" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 43385 } }
I20260806 22:49:52.443120 27869 leader_election.cc:288] T 51b0121ec03b47f1ab8f3ac3458d2a3d P fd1b2e1656e24421813074d3a7a3f6bb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91e4eeee79304b0da16d36864b9fcc1f (127.6.40.66:38635), 87fd367889cc4d379ad9e432f8572051 (127.6.40.67:46713)
W20260806 22:49:52.443563 27302 leader_election.cc:341] T 51b0121ec03b47f1ab8f3ac3458d2a3d P fd1b2e1656e24421813074d3a7a3f6bb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 91e4eeee79304b0da16d36864b9fcc1f (127.6.40.66:38635): Not found: Tablet not found: 51b0121ec03b47f1ab8f3ac3458d2a3d
W20260806 22:49:52.443670 27299 leader_election.cc:341] T 51b0121ec03b47f1ab8f3ac3458d2a3d P fd1b2e1656e24421813074d3a7a3f6bb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 87fd367889cc4d379ad9e432f8572051 (127.6.40.67:46713): Not found: Tablet not found: 51b0121ec03b47f1ab8f3ac3458d2a3d
I20260806 22:49:52.443732 27299 leader_election.cc:302] T 51b0121ec03b47f1ab8f3ac3458d2a3d P fd1b2e1656e24421813074d3a7a3f6bb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fd1b2e1656e24421813074d3a7a3f6bb; no voters: 87fd367889cc4d379ad9e432f8572051, 91e4eeee79304b0da16d36864b9fcc1f
I20260806 22:49:52.443826 27869 raft_consensus.cc:2726] T 51b0121ec03b47f1ab8f3ac3458d2a3d P fd1b2e1656e24421813074d3a7a3f6bb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 22:49:53.072479 27872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:53.072153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:32939 (local address 127.6.40.65:54807)
0806 22:49:53.072387 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:53.072426 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:32939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":66,"threads_started":1}
I20260806 22:49:53.167157 27874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:53.166943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:32939 (local address 127.6.40.66:38591)
0806 22:49:53.167069 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:53.167109 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:32939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":67,"threads_started":1}
I20260806 22:49:53.302146 27876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:53.301884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:32939 (local address 127.6.40.67:39475)
0806 22:49:53.302041 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:53.302081 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:32939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":84,"threads_started":1}
I20260806 22:49:54.073290 27879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:54.073060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:32939 (local address 127.6.40.65:44197)
0806 22:49:54.073201 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:54.073239 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:32939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":79,"threads_started":1}
I20260806 22:49:54.167829 27880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:54.167606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:32939 (local address 127.6.40.66:35217)
0806 22:49:54.167740 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:54.167781 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:32939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":66,"threads_started":1}
I20260806 22:49:54.302871 27881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:54.302650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:32939 (local address 127.6.40.67:41599)
0806 22:49:54.302785 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:54.302824 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:32939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":67,"threads_started":1}
I20260806 22:49:55.074029 27882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:55.073807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:32939 (local address 127.6.40.65:33845)
0806 22:49:55.073936 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:55.073978 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:32939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":63,"threads_started":1}
I20260806 22:49:55.168530 27883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:55.168295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:32939 (local address 127.6.40.66:45789)
0806 22:49:55.168433 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:55.168485 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:32939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":79,"threads_started":1}
I20260806 22:49:55.252717 27884 raft_consensus.cc:482] T 51b0121ec03b47f1ab8f3ac3458d2a3d P fd1b2e1656e24421813074d3a7a3f6bb [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:49:55.252880 27884 raft_consensus.cc:504] T 51b0121ec03b47f1ab8f3ac3458d2a3d P fd1b2e1656e24421813074d3a7a3f6bb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e4eeee79304b0da16d36864b9fcc1f" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 38635 } } peers { permanent_uuid: "87fd367889cc4d379ad9e432f8572051" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 46713 } } peers { permanent_uuid: "fd1b2e1656e24421813074d3a7a3f6bb" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 43385 } }
I20260806 22:49:55.253144 27884 leader_election.cc:288] T 51b0121ec03b47f1ab8f3ac3458d2a3d P fd1b2e1656e24421813074d3a7a3f6bb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91e4eeee79304b0da16d36864b9fcc1f (127.6.40.66:38635), 87fd367889cc4d379ad9e432f8572051 (127.6.40.67:46713)
W20260806 22:49:55.253541 27302 leader_election.cc:341] T 51b0121ec03b47f1ab8f3ac3458d2a3d P fd1b2e1656e24421813074d3a7a3f6bb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 91e4eeee79304b0da16d36864b9fcc1f (127.6.40.66:38635): Not found: Tablet not found: 51b0121ec03b47f1ab8f3ac3458d2a3d
W20260806 22:49:55.253651 27299 leader_election.cc:341] T 51b0121ec03b47f1ab8f3ac3458d2a3d P fd1b2e1656e24421813074d3a7a3f6bb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 87fd367889cc4d379ad9e432f8572051 (127.6.40.67:46713): Not found: Tablet not found: 51b0121ec03b47f1ab8f3ac3458d2a3d
I20260806 22:49:55.253769 27299 leader_election.cc:302] T 51b0121ec03b47f1ab8f3ac3458d2a3d P fd1b2e1656e24421813074d3a7a3f6bb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fd1b2e1656e24421813074d3a7a3f6bb; no voters: 87fd367889cc4d379ad9e432f8572051, 91e4eeee79304b0da16d36864b9fcc1f
I20260806 22:49:55.253866 27884 raft_consensus.cc:2726] T 51b0121ec03b47f1ab8f3ac3458d2a3d P fd1b2e1656e24421813074d3a7a3f6bb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 22:49:55.303660 27885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:55.303386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:32939 (local address 127.6.40.67:46025)
0806 22:49:55.303564 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:55.303609 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:32939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":86,"threads_started":1}
I20260806 22:49:56.074729 27887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:56.074474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:32939 (local address 127.6.40.65:57411)
0806 22:49:56.074629 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:56.074677 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:32939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":69,"threads_started":1}
I20260806 22:49:56.169297 27888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:56.169053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:32939 (local address 127.6.40.66:33247)
0806 22:49:56.169211 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:56.169250 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:32939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":82,"threads_started":1}
I20260806 22:49:56.304597 27889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:56.304342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:32939 (local address 127.6.40.67:34989)
0806 22:49:56.304509 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:56.304549 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:32939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":105,"threads_started":1}
I20260806 22:49:57.075511 27892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:57.075274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:32939 (local address 127.6.40.65:57863)
0806 22:49:57.075425 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:57.075463 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:32939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":90,"threads_started":1}
I20260806 22:49:57.170071 27893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:57.169806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:32939 (local address 127.6.40.66:36657)
0806 22:49:57.169966 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:57.170019 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:32939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":73,"threads_started":1}
I20260806 22:49:57.305325 27896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:57.305093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:32939 (local address 127.6.40.67:44731)
0806 22:49:57.305238 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:57.305276 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:32939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":83,"threads_started":1}
I20260806 22:49:58.076355 27897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:58.076074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:32939 (local address 127.6.40.65:55515)
0806 22:49:58.076227 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:58.076266 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:32939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":87,"threads_started":1}
I20260806 22:49:58.170885 27898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:58.170668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:32939 (local address 127.6.40.66:45507)
0806 22:49:58.170799 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:58.170836 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:32939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":70,"threads_started":1}
I20260806 22:49:58.299387 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 27240
I20260806 22:49:58.315029 27899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:58.306324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:32939 (local address 127.6.40.67:33629)
0806 22:49:58.314895 (+ 8571us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:58.314959 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:32939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":8543,"thread_start_us":380,"threads_started":1}
I20260806 22:49:58.320320 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 27456
I20260806 22:49:58.330071 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 27642
2026-08-06T22:49:58Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18690 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:49:58 dist-test-slave-zh2d krb5kdc[27902](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:49:58 dist-test-slave-zh2d krb5kdc[27902](info): set up 2 sockets
Aug 06 22:49:58 dist-test-slave-zh2d krb5kdc[27902](info): commencing operation
krb5kdc: starting...
W20260806 22:49:59.590118 6305 mini_kdc.cc:121] Time spent starting KDC: real 1.234s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/krb5kdc/test-user.keytab.
Aug 06 22:49:59 dist-test-slave-zh2d krb5kdc[27902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056599, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:49:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:49:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.40.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.40.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.40.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab.
Entry for principal kudu/127.6.40.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab.
I20260806 22:49:59.720395 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.126:42633
--webserver_interface=127.6.40.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab
--principal=kudu/127.6.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:36473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.40.126:42633
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:49:59.804593 27919 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:49:59.804796 27919 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:49:59.804817 27919 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:49:59.806062 27919 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 22:49:59.806102 27919 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 22:49:59.806115 27919 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 22:49:59.806128 27919 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:49:59.806140 27919 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:49:59.806195 27919 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:49:59.806212 27919 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:49:59.806236 27919 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:49:59.807699 27919 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:36473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.6.40.126:42633
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab
--principal=kudu/127.6.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.126:42633
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27919
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:49:59.807963 27919 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:49:59.808214 27919 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 22:49:59.813725 27919 server_base.cc:1034] running on GCE node
W20260806 22:49:59.813690 27931 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:59.813684 27928 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:59.813694 27937 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:59.814106 27919 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:49:59.814435 27919 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:49:59.815595 27919 hybrid_clock.cc:648] HybridClock initialized: now 1786056599815573 us; error 41 us; skew 500 ppm
Aug 06 22:49:59 dist-test-slave-zh2d krb5kdc[27902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056599, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:49:59.818503 27919 init.cc:377] Logged in from keytab as kudu/127.6.40.126@KRBTEST.COM (short username kudu)
I20260806 22:49:59.819046 27919 webserver.cc:460] Webserver started at http://127.6.40.126:43717/ using document root <none> and password file <none>
I20260806 22:49:59.819278 27919 fs_manager.cc:359] Metadata directory not provided
I20260806 22:49:59.819324 27919 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:49:59.819459 27919 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:49:59.820575 27919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/master-0/data/instance:
uuid: "0a7f7cb97a9a4b75b6c39701b1d3e39f"
format_stamp: "Formatted at 2026-08-06 22:49:59 on dist-test-slave-zh2d"
server_key: "b909af0f5c21d28eae37cac377290833"
server_key_iv: "a755d12d30bf58a14981a6b7044aff99"
server_key_version: "encryptionkey@0"
I20260806 22:49:59.820996 27919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/master-0/wal/instance:
uuid: "0a7f7cb97a9a4b75b6c39701b1d3e39f"
format_stamp: "Formatted at 2026-08-06 22:49:59 on dist-test-slave-zh2d"
server_key: "b909af0f5c21d28eae37cac377290833"
server_key_iv: "a755d12d30bf58a14981a6b7044aff99"
server_key_version: "encryptionkey@0"
I20260806 22:49:59.822444 27919 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:49:59.823256 27947 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:59.823484 27919 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:49:59.823598 27919 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/master-0/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/master-0/wal
uuid: "0a7f7cb97a9a4b75b6c39701b1d3e39f"
format_stamp: "Formatted at 2026-08-06 22:49:59 on dist-test-slave-zh2d"
server_key: "b909af0f5c21d28eae37cac377290833"
server_key_iv: "a755d12d30bf58a14981a6b7044aff99"
server_key_version: "encryptionkey@0"
I20260806 22:49:59.823676 27919 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:49:59.833966 27919 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:49:59.836369 27919 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:49:59.836540 27919 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:49:59.840916 27919 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.126:42633
I20260806 22:49:59.840952 28017 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.126:42633 every 8 connection(s)
I20260806 22:49:59.841307 27919 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
I20260806 22:49:59.842226 28019 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:49:59.844535 28019 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a7f7cb97a9a4b75b6c39701b1d3e39f: Bootstrap starting.
I20260806 22:49:59.845219 28019 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0a7f7cb97a9a4b75b6c39701b1d3e39f: Neither blocks nor log segments found. Creating new log.
I20260806 22:49:59.845490 28019 log.cc:824] T 00000000000000000000000000000000 P 0a7f7cb97a9a4b75b6c39701b1d3e39f: Log is configured to *not* fsync() on all Append() calls
I20260806 22:49:59.846155 28019 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a7f7cb97a9a4b75b6c39701b1d3e39f: No bootstrap required, opened a new log
I20260806 22:49:59.846570 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 27919
I20260806 22:49:59.846689 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/master-0/wal/instance
I20260806 22:49:59.846901 6305 external_mini_cluster.cc:1428] Setting key 93238525760bf8a4841de0e95d032219
I20260806 22:49:59.848393 28019 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0a7f7cb97a9a4b75b6c39701b1d3e39f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a7f7cb97a9a4b75b6c39701b1d3e39f" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 42633 } }
I20260806 22:49:59.848562 28019 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0a7f7cb97a9a4b75b6c39701b1d3e39f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:49:59.848603 28019 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0a7f7cb97a9a4b75b6c39701b1d3e39f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a7f7cb97a9a4b75b6c39701b1d3e39f, State: Initialized, Role: FOLLOWER
I20260806 22:49:59.848793 28019 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0a7f7cb97a9a4b75b6c39701b1d3e39f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a7f7cb97a9a4b75b6c39701b1d3e39f" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 42633 } }
I20260806 22:49:59.848938 28019 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0a7f7cb97a9a4b75b6c39701b1d3e39f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:49:59.848996 28019 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0a7f7cb97a9a4b75b6c39701b1d3e39f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:49:59.849054 28019 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0a7f7cb97a9a4b75b6c39701b1d3e39f [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:49:59.849779 28019 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0a7f7cb97a9a4b75b6c39701b1d3e39f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a7f7cb97a9a4b75b6c39701b1d3e39f" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 42633 } }
I20260806 22:49:59.849876 28019 leader_election.cc:302] T 00000000000000000000000000000000 P 0a7f7cb97a9a4b75b6c39701b1d3e39f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0a7f7cb97a9a4b75b6c39701b1d3e39f; no voters:
I20260806 22:49:59.850095 28019 leader_election.cc:288] T 00000000000000000000000000000000 P 0a7f7cb97a9a4b75b6c39701b1d3e39f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:49:59.850147 28026 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0a7f7cb97a9a4b75b6c39701b1d3e39f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:49:59.850330 28026 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0a7f7cb97a9a4b75b6c39701b1d3e39f [term 1 LEADER]: Becoming Leader. State: Replica: 0a7f7cb97a9a4b75b6c39701b1d3e39f, State: Running, Role: LEADER
I20260806 22:49:59.850368 28019 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0a7f7cb97a9a4b75b6c39701b1d3e39f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:49:59.850716 28026 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0a7f7cb97a9a4b75b6c39701b1d3e39f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a7f7cb97a9a4b75b6c39701b1d3e39f" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 42633 } }
I20260806 22:49:59.851128 28028 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a7f7cb97a9a4b75b6c39701b1d3e39f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0a7f7cb97a9a4b75b6c39701b1d3e39f. Latest consensus state: current_term: 1 leader_uuid: "0a7f7cb97a9a4b75b6c39701b1d3e39f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a7f7cb97a9a4b75b6c39701b1d3e39f" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 42633 } } }
I20260806 22:49:59.851145 28027 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a7f7cb97a9a4b75b6c39701b1d3e39f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0a7f7cb97a9a4b75b6c39701b1d3e39f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a7f7cb97a9a4b75b6c39701b1d3e39f" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 42633 } } }
I20260806 22:49:59.851303 28028 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a7f7cb97a9a4b75b6c39701b1d3e39f [sys.catalog]: This master's current role is: LEADER
I20260806 22:49:59.851316 28027 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a7f7cb97a9a4b75b6c39701b1d3e39f [sys.catalog]: This master's current role is: LEADER
I20260806 22:49:59.851576 28037 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:49:59.852128 28037 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:49:59.854718 28037 catalog_manager.cc:1347] Generated new cluster ID: b6dca6ddc3014539b3d05e506340117b
I20260806 22:49:59.854790 28037 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 22:49:59 dist-test-slave-zh2d krb5kdc[27902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056599, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
I20260806 22:49:59.859647 28024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:59.847448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59160 (local address 127.6.40.126:42633)
0806 22:49:59.847662 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:59.847677 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:59.847703 (+ 26us) server_negotiation.cc:407] Connection header received
0806 22:49:59.848376 (+ 673us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:59.848382 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:59.848430 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:59.848586 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:59.849470 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:59.851253 (+ 1783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:59.852386 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:59.852995 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:59.853150 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:59.855910 (+ 2760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:59.855914 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:59.855914 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:59.855925 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:59.857951 (+ 2026us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:59.858394 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:59.858397 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:59.858397 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:59.858525 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:59.858763 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:59.858764 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:59.858764 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:59.859042 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:59.859163 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:59.859315 (+ 152us) server_negotiation.cc:299] Negotiation successful
0806 22:49:59.859441 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":114,"threads_started":1}
I20260806 22:49:59.867707 28037 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:49:59.868216 28037 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:49:59.872794 28037 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0a7f7cb97a9a4b75b6c39701b1d3e39f: Generated new TSK 0
I20260806 22:49:59.872983 28037 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.40.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.40.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.40.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.65.keytab.
Entry for principal kudu/127.6.40.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.65.keytab.
I20260806 22:49:59.907655 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.65:0
--local_ip_for_outbound_sockets=127.6.40.65
--webserver_interface=127.6.40.65
--webserver_port=0
--tserver_master_addrs=127.6.40.126:42633
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.65.keytab
--principal=kudu/127.6.40.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:36473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:49:59.993733 28054 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:49:59.993932 28054 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:49:59.993955 28054 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:49:59.995105 28054 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:49:59.995151 28054 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:49:59.995177 28054 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.65
I20260806 22:49:59.996662 28054 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:36473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.65.keytab
--principal=kudu/127.6.40.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.40.65
--webserver_port=0
--tserver_master_addrs=127.6.40.126:42633
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28054
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.65
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:49:59.996932 28054 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:49:59.997195 28054 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:50:00.002557 28062 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:00.002563 28061 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:00.002563 28064 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:00.002873 28054 server_base.cc:1034] running on GCE node
I20260806 22:50:00.003118 28054 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:50:00.003399 28054 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:50:00.004585 28054 hybrid_clock.cc:648] HybridClock initialized: now 1786056600004569 us; error 47 us; skew 500 ppm
Aug 06 22:50:00 dist-test-slave-zh2d krb5kdc[27902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056600, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:50:00.007488 28054 init.cc:377] Logged in from keytab as kudu/127.6.40.65@KRBTEST.COM (short username kudu)
I20260806 22:50:00.007959 28054 webserver.cc:460] Webserver started at http://127.6.40.65:34111/ using document root <none> and password file <none>
I20260806 22:50:00.008206 28054 fs_manager.cc:359] Metadata directory not provided
I20260806 22:50:00.008275 28054 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:50:00.008438 28054 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:50:00.009611 28054 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-0/data/instance:
uuid: "3a75f07037a54584a1b8d1b3bd13336a"
format_stamp: "Formatted at 2026-08-06 22:50:00 on dist-test-slave-zh2d"
server_key: "25aba12f593f79340ee874c9a3eb7765"
server_key_iv: "e23616f1a04dbb5dbf1f790b1201bbde"
server_key_version: "encryptionkey@0"
I20260806 22:50:00.010031 28054 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-0/wal/instance:
uuid: "3a75f07037a54584a1b8d1b3bd13336a"
format_stamp: "Formatted at 2026-08-06 22:50:00 on dist-test-slave-zh2d"
server_key: "25aba12f593f79340ee874c9a3eb7765"
server_key_iv: "e23616f1a04dbb5dbf1f790b1201bbde"
server_key_version: "encryptionkey@0"
I20260806 22:50:00.011472 28054 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:50:00.012336 28075 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:00.012583 28054 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:50:00.012701 28054 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-0/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-0/wal
uuid: "3a75f07037a54584a1b8d1b3bd13336a"
format_stamp: "Formatted at 2026-08-06 22:50:00 on dist-test-slave-zh2d"
server_key: "25aba12f593f79340ee874c9a3eb7765"
server_key_iv: "e23616f1a04dbb5dbf1f790b1201bbde"
server_key_version: "encryptionkey@0"
I20260806 22:50:00.012777 28054 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:50:00.024544 28054 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:50:00.027074 28054 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:50:00.027247 28054 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:50:00.027522 28054 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:50:00.027892 28054 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:50:00.027930 28054 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:00.027953 28054 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:50:00.027988 28054 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:00.034416 28054 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.65:37891
I20260806 22:50:00.034492 28228 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.65:37891 every 8 connection(s)
I20260806 22:50:00.034825 28054 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
Aug 06 22:50:00 dist-test-slave-zh2d krb5kdc[27902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056600, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.65@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
I20260806 22:50:00.043709 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 28054
I20260806 22:50:00.043833 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-0/wal/instance
I20260806 22:50:00.044090 6305 external_mini_cluster.cc:1428] Setting key 0f818b057315531e24c25ee389c15d4f
I20260806 22:50:00.045202 28024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:00.035684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.65:51815 (local address 127.6.40.126:42633)
0806 22:50:00.035767 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:00.035769 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:00.035990 (+ 221us) server_negotiation.cc:407] Connection header received
0806 22:50:00.036453 (+ 463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:00.036454 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:00.036468 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:00.036541 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:00.037534 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:00.038045 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:00.039062 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:00.039271 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:00.039380 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:00.041917 (+ 2537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:00.041919 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:00.041919 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:00.041935 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:00.043467 (+ 1532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:00.043969 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:00.043970 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:00.043971 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:00.044044 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:00.044386 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:00.044387 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:00.044387 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:00.044509 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:00.044570 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:00.044950 (+ 380us) server_negotiation.cc:299] Negotiation successful
0806 22:50:00.045054 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 22:50:00.045284 28232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:00.035638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:42633 (local address 127.6.40.65:51815)
0806 22:50:00.035836 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:00.035878 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:00.036360 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:00.036596 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:00.036599 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:00.037248 (+ 649us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:00.037445 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:00.037450 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:00.038133 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:00.038134 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:00.038969 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:00.038972 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:00.039398 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:00.039399 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:00.039575 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:00.040112 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:00.040123 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:00.041800 (+ 1677us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:50:00.043567 (+ 1767us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:00.043577 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:00.043579 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:00.043856 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:00.044126 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:00.044127 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:00.044127 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:00.044308 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:00.044567 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:50:00.044575 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:50:00.044736 (+ 161us) client_negotiation.cc:769] Sending connection context
0806 22:50:00.044926 (+ 190us) client_negotiation.cc:240] Negotiation successful
0806 22:50:00.045088 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":108,"threads_started":1}
I20260806 22:50:00.047207 28229 heartbeater.cc:347] Connected to a master server at 127.6.40.126:42633
I20260806 22:50:00.047303 28229 heartbeater.cc:464] Registering TS with master...
I20260806 22:50:00.047499 28229 heartbeater.cc:511] Master 127.6.40.126:42633 requested a full tablet report, sending...
I20260806 22:50:00.048049 27973 ts_manager.cc:194] Registered new tserver with Master: 3a75f07037a54584a1b8d1b3bd13336a (127.6.40.65:37891)
I20260806 22:50:00.048663 27973 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.40.65@KRBTEST.COM'} at 127.6.40.65:51815
WARNING: no policy specified for kudu/127.6.40.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.40.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.40.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.66.keytab.
Entry for principal kudu/127.6.40.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.66.keytab.
I20260806 22:50:00.072082 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.66:0
--local_ip_for_outbound_sockets=127.6.40.66
--webserver_interface=127.6.40.66
--webserver_port=0
--tserver_master_addrs=127.6.40.126:42633
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.66.keytab
--principal=kudu/127.6.40.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:36473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:50:00.156669 28236 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:50:00.156960 28236 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:50:00.156991 28236 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:50:00.158119 28236 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:50:00.158156 28236 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:50:00.158182 28236 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.66
I20260806 22:50:00.159533 28236 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:36473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.66.keytab
--principal=kudu/127.6.40.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.40.66
--webserver_port=0
--tserver_master_addrs=127.6.40.126:42633
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28236
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.66
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:50:00.159708 28236 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:50:00.159885 28236 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:50:00.164973 28242 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:00.165123 28236 server_base.cc:1034] running on GCE node
W20260806 22:50:00.164991 28246 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:00.164991 28241 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:00.165652 28236 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:50:00.165934 28236 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:50:00.167122 28236 hybrid_clock.cc:648] HybridClock initialized: now 1786056600167106 us; error 38 us; skew 500 ppm
Aug 06 22:50:00 dist-test-slave-zh2d krb5kdc[27902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056600, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:50:00.169844 28236 init.cc:377] Logged in from keytab as kudu/127.6.40.66@KRBTEST.COM (short username kudu)
I20260806 22:50:00.170292 28236 webserver.cc:460] Webserver started at http://127.6.40.66:33437/ using document root <none> and password file <none>
I20260806 22:50:00.170516 28236 fs_manager.cc:359] Metadata directory not provided
I20260806 22:50:00.170585 28236 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:50:00.170737 28236 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:50:00.171808 28236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-1/data/instance:
uuid: "a152c25bb71949a0a62dce1e0476c6d3"
format_stamp: "Formatted at 2026-08-06 22:50:00 on dist-test-slave-zh2d"
server_key: "9b0dcf3af3447da2fadf70f640bed9a7"
server_key_iv: "bd8b2fbc787a5c9153913649d60b0be1"
server_key_version: "encryptionkey@0"
I20260806 22:50:00.172185 28236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-1/wal/instance:
uuid: "a152c25bb71949a0a62dce1e0476c6d3"
format_stamp: "Formatted at 2026-08-06 22:50:00 on dist-test-slave-zh2d"
server_key: "9b0dcf3af3447da2fadf70f640bed9a7"
server_key_iv: "bd8b2fbc787a5c9153913649d60b0be1"
server_key_version: "encryptionkey@0"
I20260806 22:50:00.173719 28236 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:50:00.174618 28255 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:00.174834 28236 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:50:00.174948 28236 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-1/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-1/wal
uuid: "a152c25bb71949a0a62dce1e0476c6d3"
format_stamp: "Formatted at 2026-08-06 22:50:00 on dist-test-slave-zh2d"
server_key: "9b0dcf3af3447da2fadf70f640bed9a7"
server_key_iv: "bd8b2fbc787a5c9153913649d60b0be1"
server_key_version: "encryptionkey@0"
I20260806 22:50:00.175037 28236 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:50:00.186849 28236 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:50:00.189339 28236 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:50:00.189515 28236 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:50:00.189760 28236 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:50:00.190071 28236 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:50:00.190125 28236 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:00.190161 28236 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:50:00.190191 28236 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:00.196439 28236 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.66:37655
I20260806 22:50:00.196493 28416 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.66:37655 every 8 connection(s)
I20260806 22:50:00.196815 28236 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
I20260806 22:50:00.197503 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 28236
I20260806 22:50:00.197618 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-1/wal/instance
I20260806 22:50:00.197846 6305 external_mini_cluster.cc:1428] Setting key b127e510d96e5788d0f55adc6a94f38d
Aug 06 22:50:00 dist-test-slave-zh2d krb5kdc[27902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056600, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.66@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.40.67@KRBTEST.COM; defaulting to no policy
I20260806 22:50:00.207588 28024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:00.197692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.66:55745 (local address 127.6.40.126:42633)
0806 22:50:00.197763 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:00.197765 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:00.198001 (+ 236us) server_negotiation.cc:407] Connection header received
0806 22:50:00.198627 (+ 626us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:00.198628 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:00.198640 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:00.198688 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:00.199672 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:00.200178 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:00.201759 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:00.201966 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:00.202030 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:00.204506 (+ 2476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:00.204508 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:00.204508 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:00.204517 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:00.206045 (+ 1528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:00.206489 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:00.206491 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:00.206491 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:00.206557 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:00.206825 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:00.206826 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:00.206826 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:00.206958 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:00.207032 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:00.207367 (+ 335us) server_negotiation.cc:299] Negotiation successful
0806 22:50:00.207479 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 22:50:00.207873 28420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:00.197638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:42633 (local address 127.6.40.66:55745)
0806 22:50:00.197851 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:00.197914 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:00.198526 (+ 612us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:00.198761 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:00.198766 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:00.199365 (+ 599us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:00.199558 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:00.199563 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:00.200284 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:00.200285 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:00.201657 (+ 1372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:00.201660 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:00.202063 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:00.202064 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:00.202215 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:00.202739 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:00.202750 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:00.204405 (+ 1655us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:50:00.206159 (+ 1754us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:00.206168 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:00.206170 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:00.206410 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:00.206628 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:00.206629 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:00.206630 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:00.206749 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:00.207036 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:50:00.207044 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:50:00.207210 (+ 166us) client_negotiation.cc:769] Sending connection context
0806 22:50:00.207345 (+ 135us) client_negotiation.cc:240] Negotiation successful
0806 22:50:00.207561 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":99,"threads_started":1}
I20260806 22:50:00.208544 28417 heartbeater.cc:347] Connected to a master server at 127.6.40.126:42633
I20260806 22:50:00.208660 28417 heartbeater.cc:464] Registering TS with master...
I20260806 22:50:00.208925 28417 heartbeater.cc:511] Master 127.6.40.126:42633 requested a full tablet report, sending...
I20260806 22:50:00.209421 27973 ts_manager.cc:194] Registered new tserver with Master: a152c25bb71949a0a62dce1e0476c6d3 (127.6.40.66:37655)
I20260806 22:50:00.209925 27973 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.40.66@KRBTEST.COM'} at 127.6.40.66:55745
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.40.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.40.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.67.keytab.
Entry for principal kudu/127.6.40.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.67.keytab.
I20260806 22:50:00.227986 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.67:0
--local_ip_for_outbound_sockets=127.6.40.67
--webserver_interface=127.6.40.67
--webserver_port=0
--tserver_master_addrs=127.6.40.126:42633
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.67.keytab
--principal=kudu/127.6.40.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:36473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:50:00.312659 28425 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:50:00.312955 28425 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:50:00.312986 28425 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:50:00.314118 28425 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:50:00.314155 28425 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:50:00.314181 28425 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.67
I20260806 22:50:00.315719 28425 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:36473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.67.keytab
--principal=kudu/127.6.40.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.40.67
--webserver_port=0
--tserver_master_addrs=127.6.40.126:42633
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28425
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.67
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:50:00.315965 28425 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:50:00.316215 28425 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:50:00.321506 28437 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:00.321512 28432 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:00.322556 28425 server_base.cc:1034] running on GCE node
W20260806 22:50:00.321527 28433 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:00.322863 28425 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:50:00.323120 28425 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:50:00.324299 28425 hybrid_clock.cc:648] HybridClock initialized: now 1786056600324285 us; error 33 us; skew 500 ppm
Aug 06 22:50:00 dist-test-slave-zh2d krb5kdc[27902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056600, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:50:00.327189 28425 init.cc:377] Logged in from keytab as kudu/127.6.40.67@KRBTEST.COM (short username kudu)
I20260806 22:50:00.327788 28425 webserver.cc:460] Webserver started at http://127.6.40.67:46549/ using document root <none> and password file <none>
I20260806 22:50:00.328065 28425 fs_manager.cc:359] Metadata directory not provided
I20260806 22:50:00.328135 28425 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:50:00.328300 28425 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:50:00.329584 28425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-2/data/instance:
uuid: "baf570133f8749b68b951eaf0fcd9758"
format_stamp: "Formatted at 2026-08-06 22:50:00 on dist-test-slave-zh2d"
server_key: "65b9da73df9edc56555e857cec9777c2"
server_key_iv: "91a59b4ea6bcf5c50a1dd0b7036f0620"
server_key_version: "encryptionkey@0"
I20260806 22:50:00.329986 28425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-2/wal/instance:
uuid: "baf570133f8749b68b951eaf0fcd9758"
format_stamp: "Formatted at 2026-08-06 22:50:00 on dist-test-slave-zh2d"
server_key: "65b9da73df9edc56555e857cec9777c2"
server_key_iv: "91a59b4ea6bcf5c50a1dd0b7036f0620"
server_key_version: "encryptionkey@0"
I20260806 22:50:00.331517 28425 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:50:00.332496 28447 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:00.332754 28425 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:50:00.332904 28425 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-2/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-2/wal
uuid: "baf570133f8749b68b951eaf0fcd9758"
format_stamp: "Formatted at 2026-08-06 22:50:00 on dist-test-slave-zh2d"
server_key: "65b9da73df9edc56555e857cec9777c2"
server_key_iv: "91a59b4ea6bcf5c50a1dd0b7036f0620"
server_key_version: "encryptionkey@0"
I20260806 22:50:00.333004 28425 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:50:00.350688 28425 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:50:00.353372 28425 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:50:00.353549 28425 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:50:00.353816 28425 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:50:00.354171 28425 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:50:00.354226 28425 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:00.354298 28425 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:50:00.354331 28425 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:00.361490 28425 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.67:32857
I20260806 22:50:00.361543 28599 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.67:32857 every 8 connection(s)
I20260806 22:50:00.361927 28425 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
I20260806 22:50:00.363698 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 28425
I20260806 22:50:00.363793 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-2/wal/instance
I20260806 22:50:00.363979 6305 external_mini_cluster.cc:1428] Setting key 4f93f059f5b4f67c7f74af56c6bd5de8
Aug 06 22:50:00 dist-test-slave-zh2d krb5kdc[27902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056600, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.67@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
I20260806 22:50:00.372164 28024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:00.362753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.67:35587 (local address 127.6.40.126:42633)
0806 22:50:00.362836 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:00.362838 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:00.363003 (+ 165us) server_negotiation.cc:407] Connection header received
0806 22:50:00.363554 (+ 551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:00.363556 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:00.363569 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:00.363647 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:00.364969 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:00.365537 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:00.366599 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:00.366789 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:00.366847 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:00.369271 (+ 2424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:00.369272 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:00.369273 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:00.369280 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:00.370658 (+ 1378us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:00.371065 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:00.371066 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:00.371066 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:00.371117 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:00.371504 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:00.371505 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:00.371505 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:00.371632 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:00.371703 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:00.372021 (+ 318us) server_negotiation.cc:299] Negotiation successful
0806 22:50:00.372093 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 22:50:00.372383 28603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:00.362685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:42633 (local address 127.6.40.67:35587)
0806 22:50:00.362878 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:00.362917 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:00.363446 (+ 529us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:00.363699 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:00.363702 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:00.364504 (+ 802us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:00.364824 (+ 320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:00.364833 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:00.365642 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:00.365643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:00.366514 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:00.366517 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:00.366870 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:00.366871 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:00.366972 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:00.367545 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:00.367554 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:00.369178 (+ 1624us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:50:00.370749 (+ 1571us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:00.370756 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:00.370758 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:00.370996 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:00.371209 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:00.371210 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:00.371211 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:00.371405 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:00.371729 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:50:00.371737 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:50:00.371868 (+ 131us) client_negotiation.cc:769] Sending connection context
0806 22:50:00.371969 (+ 101us) client_negotiation.cc:240] Negotiation successful
0806 22:50:00.372112 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":104,"threads_started":1}
I20260806 22:50:00.373059 28600 heartbeater.cc:347] Connected to a master server at 127.6.40.126:42633
I20260806 22:50:00.373157 28600 heartbeater.cc:464] Registering TS with master...
I20260806 22:50:00.373353 28600 heartbeater.cc:511] Master 127.6.40.126:42633 requested a full tablet report, sending...
I20260806 22:50:00.373813 27973 ts_manager.cc:194] Registered new tserver with Master: baf570133f8749b68b951eaf0fcd9758 (127.6.40.67:32857)
I20260806 22:50:00.374233 27973 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.40.67@KRBTEST.COM'} at 127.6.40.67:35587
I20260806 22:50:00.375965 6305 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:50:00.383688 28024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:00.377391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59162 (local address 127.6.40.126:42633)
0806 22:50:00.377431 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:00.377433 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:00.377555 (+ 122us) server_negotiation.cc:407] Connection header received
0806 22:50:00.377609 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:00.377609 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:00.377618 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:00.377663 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:00.378277 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:00.378766 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:00.379599 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:00.379781 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:00.379837 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:00.381194 (+ 1357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:00.381195 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:00.381195 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:00.381200 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:00.382530 (+ 1330us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:00.382915 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:00.382916 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:00.382916 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:00.382971 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:00.383121 (+ 150us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:00.383122 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:00.383122 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:00.383273 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:00.383352 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:00.383546 (+ 194us) server_negotiation.cc:299] Negotiation successful
0806 22:50:00.383622 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 22:50:00.385612 27973 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59162:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:50:00.387322 27973 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:50:00.394768 28623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:00.390436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47258 (local address 127.6.40.66:37655)
0806 22:50:00.390659 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:00.390664 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:00.391135 (+ 471us) server_negotiation.cc:407] Connection header received
0806 22:50:00.391181 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:00.391183 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:00.391198 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:00.391275 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:50:00.391460 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:00.392516 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:00.393596 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:00.394307 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:00.394395 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:00.394457 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:00.394632 (+ 175us) server_negotiation.cc:299] Negotiation successful
0806 22:50:00.394676 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":105,"threads_started":1}
I20260806 22:50:00.394761 28620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:00.390265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.66:37655 (local address 127.0.0.1:47258)
0806 22:50:00.391009 (+ 744us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:00.391020 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:00.391076 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:00.391306 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:00.391307 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:00.391309 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:50:00.391361 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:00.391363 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:00.392621 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:00.392622 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:00.393499 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:00.393502 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:00.394406 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:00.394408 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:00.394494 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:00.394512 (+ 18us) client_negotiation.cc:769] Sending connection context
0806 22:50:00.394605 (+ 93us) client_negotiation.cc:240] Negotiation successful
0806 22:50:00.394665 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":724,"thread_start_us":94,"threads_started":1}
I20260806 22:50:00.395215 28615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:00.390025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.65:37891 (local address 127.0.0.1:54562)
0806 22:50:00.390370 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:00.390455 (+ 85us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:00.390522 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:00.390912 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:00.390916 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:00.390919 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:50:00.390994 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:00.390996 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:00.392409 (+ 1413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:00.392414 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:00.393708 (+ 1294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:00.393712 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:00.394948 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:00.394950 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:00.395043 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:00.395048 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:50:00.395098 (+ 50us) client_negotiation.cc:240] Negotiation successful
0806 22:50:00.395146 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"mutex_wait_us":7,"thread_start_us":63,"threads_started":1}
I20260806 22:50:00.395434 28614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:00.390032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.67:32857 (local address 127.0.0.1:41630)
0806 22:50:00.390421 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:00.390461 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:00.390543 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:00.391017 (+ 474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:00.391018 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:00.391021 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:50:00.391078 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:00.391080 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:00.392228 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:00.392230 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:00.394020 (+ 1790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:00.394023 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:00.395042 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:00.395043 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:00.395122 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:00.395128 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 22:50:00.395276 (+ 148us) client_negotiation.cc:240] Negotiation successful
0806 22:50:00.395343 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":100,"threads_started":1}
I20260806 22:50:00.395517 28619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:00.390165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54562 (local address 127.6.40.65:37891)
0806 22:50:00.390723 (+ 558us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:00.390726 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:00.390736 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:50:00.390798 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:00.390800 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:00.390816 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:00.390900 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:50:00.391092 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:00.392293 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:00.393816 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:00.394539 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:00.394632 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:00.394692 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:00.395375 (+ 683us) server_negotiation.cc:299] Negotiation successful
0806 22:50:00.395431 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":519,"thread_start_us":143,"threads_started":1}
I20260806 22:50:00.395709 28618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:00.390145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41630 (local address 127.6.40.67:32857)
0806 22:50:00.390704 (+ 559us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:00.390709 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:00.390722 (+ 13us) server_negotiation.cc:407] Connection header received
0806 22:50:00.390775 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:00.390778 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:00.390807 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:00.390913 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:50:00.391183 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:00.392103 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:00.394123 (+ 2020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:00.394911 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:00.395038 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:00.395102 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:00.395556 (+ 454us) server_negotiation.cc:299] Negotiation successful
0806 22:50:00.395612 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":507,"thread_start_us":102,"threads_started":1}
I20260806 22:50:00.395967 28328 tablet_service.cc:1467] Processing CreateTablet for tablet 8da3881d0165434388d5f6fd4604a57f (DEFAULT_TABLE table=test-table [id=e682718ffa3c4fe0b840ce13f3ac375f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:50:00.396219 28147 tablet_service.cc:1467] Processing CreateTablet for tablet 8da3881d0165434388d5f6fd4604a57f (DEFAULT_TABLE table=test-table [id=e682718ffa3c4fe0b840ce13f3ac375f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:50:00.396370 28328 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8da3881d0165434388d5f6fd4604a57f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:50:00.396638 28147 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8da3881d0165434388d5f6fd4604a57f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:50:00.396610 28517 tablet_service.cc:1467] Processing CreateTablet for tablet 8da3881d0165434388d5f6fd4604a57f (DEFAULT_TABLE table=test-table [id=e682718ffa3c4fe0b840ce13f3ac375f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:50:00.397030 28517 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8da3881d0165434388d5f6fd4604a57f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:50:00.399360 28625 tablet_bootstrap.cc:492] T 8da3881d0165434388d5f6fd4604a57f P a152c25bb71949a0a62dce1e0476c6d3: Bootstrap starting.
I20260806 22:50:00.400066 28625 tablet_bootstrap.cc:654] T 8da3881d0165434388d5f6fd4604a57f P a152c25bb71949a0a62dce1e0476c6d3: Neither blocks nor log segments found. Creating new log.
I20260806 22:50:00.400599 28625 log.cc:824] T 8da3881d0165434388d5f6fd4604a57f P a152c25bb71949a0a62dce1e0476c6d3: Log is configured to *not* fsync() on all Append() calls
I20260806 22:50:00.400960 28627 tablet_bootstrap.cc:492] T 8da3881d0165434388d5f6fd4604a57f P baf570133f8749b68b951eaf0fcd9758: Bootstrap starting.
I20260806 22:50:00.401657 28627 tablet_bootstrap.cc:654] T 8da3881d0165434388d5f6fd4604a57f P baf570133f8749b68b951eaf0fcd9758: Neither blocks nor log segments found. Creating new log.
I20260806 22:50:00.401654 28625 tablet_bootstrap.cc:492] T 8da3881d0165434388d5f6fd4604a57f P a152c25bb71949a0a62dce1e0476c6d3: No bootstrap required, opened a new log
I20260806 22:50:00.401763 28625 ts_tablet_manager.cc:1397] T 8da3881d0165434388d5f6fd4604a57f P a152c25bb71949a0a62dce1e0476c6d3: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 22:50:00.402168 28627 log.cc:824] T 8da3881d0165434388d5f6fd4604a57f P baf570133f8749b68b951eaf0fcd9758: Log is configured to *not* fsync() on all Append() calls
I20260806 22:50:00.402487 28628 tablet_bootstrap.cc:492] T 8da3881d0165434388d5f6fd4604a57f P 3a75f07037a54584a1b8d1b3bd13336a: Bootstrap starting.
I20260806 22:50:00.403045 28627 tablet_bootstrap.cc:492] T 8da3881d0165434388d5f6fd4604a57f P baf570133f8749b68b951eaf0fcd9758: No bootstrap required, opened a new log
I20260806 22:50:00.403121 28627 ts_tablet_manager.cc:1397] T 8da3881d0165434388d5f6fd4604a57f P baf570133f8749b68b951eaf0fcd9758: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:50:00.403244 28628 tablet_bootstrap.cc:654] T 8da3881d0165434388d5f6fd4604a57f P 3a75f07037a54584a1b8d1b3bd13336a: Neither blocks nor log segments found. Creating new log.
I20260806 22:50:00.403842 28628 log.cc:824] T 8da3881d0165434388d5f6fd4604a57f P 3a75f07037a54584a1b8d1b3bd13336a: Log is configured to *not* fsync() on all Append() calls
I20260806 22:50:00.403856 28625 raft_consensus.cc:348] T 8da3881d0165434388d5f6fd4604a57f P a152c25bb71949a0a62dce1e0476c6d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baf570133f8749b68b951eaf0fcd9758" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 32857 } } peers { permanent_uuid: "3a75f07037a54584a1b8d1b3bd13336a" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 37891 } } peers { permanent_uuid: "a152c25bb71949a0a62dce1e0476c6d3" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 37655 } }
I20260806 22:50:00.404101 28625 raft_consensus.cc:374] T 8da3881d0165434388d5f6fd4604a57f P a152c25bb71949a0a62dce1e0476c6d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:50:00.404156 28625 raft_consensus.cc:729] T 8da3881d0165434388d5f6fd4604a57f P a152c25bb71949a0a62dce1e0476c6d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a152c25bb71949a0a62dce1e0476c6d3, State: Initialized, Role: FOLLOWER
I20260806 22:50:00.404330 28625 consensus_queue.cc:260] T 8da3881d0165434388d5f6fd4604a57f P a152c25bb71949a0a62dce1e0476c6d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baf570133f8749b68b951eaf0fcd9758" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 32857 } } peers { permanent_uuid: "3a75f07037a54584a1b8d1b3bd13336a" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 37891 } } peers { permanent_uuid: "a152c25bb71949a0a62dce1e0476c6d3" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 37655 } }
I20260806 22:50:00.404703 28625 ts_tablet_manager.cc:1428] T 8da3881d0165434388d5f6fd4604a57f P a152c25bb71949a0a62dce1e0476c6d3: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260806 22:50:00.404924 28417 heartbeater.cc:503] Master 127.6.40.126:42633 was elected leader, sending a full tablet report...
I20260806 22:50:00.405133 28628 tablet_bootstrap.cc:492] T 8da3881d0165434388d5f6fd4604a57f P 3a75f07037a54584a1b8d1b3bd13336a: No bootstrap required, opened a new log
I20260806 22:50:00.405228 28628 ts_tablet_manager.cc:1397] T 8da3881d0165434388d5f6fd4604a57f P 3a75f07037a54584a1b8d1b3bd13336a: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 22:50:00.405426 28627 raft_consensus.cc:348] T 8da3881d0165434388d5f6fd4604a57f P baf570133f8749b68b951eaf0fcd9758 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baf570133f8749b68b951eaf0fcd9758" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 32857 } } peers { permanent_uuid: "3a75f07037a54584a1b8d1b3bd13336a" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 37891 } } peers { permanent_uuid: "a152c25bb71949a0a62dce1e0476c6d3" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 37655 } }
I20260806 22:50:00.405596 28627 raft_consensus.cc:374] T 8da3881d0165434388d5f6fd4604a57f P baf570133f8749b68b951eaf0fcd9758 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:50:00.405653 28627 raft_consensus.cc:729] T 8da3881d0165434388d5f6fd4604a57f P baf570133f8749b68b951eaf0fcd9758 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: baf570133f8749b68b951eaf0fcd9758, State: Initialized, Role: FOLLOWER
I20260806 22:50:00.405777 28627 consensus_queue.cc:260] T 8da3881d0165434388d5f6fd4604a57f P baf570133f8749b68b951eaf0fcd9758 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baf570133f8749b68b951eaf0fcd9758" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 32857 } } peers { permanent_uuid: "3a75f07037a54584a1b8d1b3bd13336a" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 37891 } } peers { permanent_uuid: "a152c25bb71949a0a62dce1e0476c6d3" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 37655 } }
I20260806 22:50:00.406116 28627 ts_tablet_manager.cc:1428] T 8da3881d0165434388d5f6fd4604a57f P baf570133f8749b68b951eaf0fcd9758: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:50:00.406116 28600 heartbeater.cc:503] Master 127.6.40.126:42633 was elected leader, sending a full tablet report...
I20260806 22:50:00.407478 28628 raft_consensus.cc:348] T 8da3881d0165434388d5f6fd4604a57f P 3a75f07037a54584a1b8d1b3bd13336a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baf570133f8749b68b951eaf0fcd9758" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 32857 } } peers { permanent_uuid: "3a75f07037a54584a1b8d1b3bd13336a" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 37891 } } peers { permanent_uuid: "a152c25bb71949a0a62dce1e0476c6d3" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 37655 } }
I20260806 22:50:00.407656 28628 raft_consensus.cc:374] T 8da3881d0165434388d5f6fd4604a57f P 3a75f07037a54584a1b8d1b3bd13336a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:50:00.407711 28628 raft_consensus.cc:729] T 8da3881d0165434388d5f6fd4604a57f P 3a75f07037a54584a1b8d1b3bd13336a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a75f07037a54584a1b8d1b3bd13336a, State: Initialized, Role: FOLLOWER
I20260806 22:50:00.407858 28628 consensus_queue.cc:260] T 8da3881d0165434388d5f6fd4604a57f P 3a75f07037a54584a1b8d1b3bd13336a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baf570133f8749b68b951eaf0fcd9758" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 32857 } } peers { permanent_uuid: "3a75f07037a54584a1b8d1b3bd13336a" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 37891 } } peers { permanent_uuid: "a152c25bb71949a0a62dce1e0476c6d3" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 37655 } }
I20260806 22:50:00.408177 28628 ts_tablet_manager.cc:1428] T 8da3881d0165434388d5f6fd4604a57f P 3a75f07037a54584a1b8d1b3bd13336a: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260806 22:50:00.408215 28229 heartbeater.cc:503] Master 127.6.40.126:42633 was elected leader, sending a full tablet report...
W20260806 22:50:00.447484 28418 tablet.cc:2369] T 8da3881d0165434388d5f6fd4604a57f P a152c25bb71949a0a62dce1e0476c6d3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:50:00.470088 28639 raft_consensus.cc:482] T 8da3881d0165434388d5f6fd4604a57f P 3a75f07037a54584a1b8d1b3bd13336a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:50:00.470221 28639 raft_consensus.cc:504] T 8da3881d0165434388d5f6fd4604a57f P 3a75f07037a54584a1b8d1b3bd13336a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baf570133f8749b68b951eaf0fcd9758" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 32857 } } peers { permanent_uuid: "3a75f07037a54584a1b8d1b3bd13336a" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 37891 } } peers { permanent_uuid: "a152c25bb71949a0a62dce1e0476c6d3" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 37655 } }
I20260806 22:50:00.470548 28639 leader_election.cc:288] T 8da3881d0165434388d5f6fd4604a57f P 3a75f07037a54584a1b8d1b3bd13336a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers baf570133f8749b68b951eaf0fcd9758 (127.6.40.67:32857), a152c25bb71949a0a62dce1e0476c6d3 (127.6.40.66:37655)
I20260806 22:50:00.474851 28618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:00.470941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.65:52987 (local address 127.6.40.67:32857)
0806 22:50:00.471002 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:00.471005 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:00.471017 (+ 12us) server_negotiation.cc:407] Connection header received
0806 22:50:00.471050 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:00.471051 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:00.471067 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:00.471130 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:50:00.471339 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:00.472207 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:00.473783 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:00.474393 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:00.474481 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:00.474495 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:00.474709 (+ 214us) server_negotiation.cc:299] Negotiation successful
0806 22:50:00.474758 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 22:50:00.474835 28623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:00.471031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.65:35455 (local address 127.6.40.66:37655)
0806 22:50:00.471082 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:00.471084 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:00.471250 (+ 166us) server_negotiation.cc:407] Connection header received
0806 22:50:00.471281 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:00.471282 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:00.471319 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:00.471420 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:50:00.471537 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:00.472305 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:00.473785 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:00.474447 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:00.474512 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:00.474523 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:00.474711 (+ 188us) server_negotiation.cc:299] Negotiation successful
0806 22:50:00.474758 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 22:50:00.474844 28232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:00.470772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.67:32857 (local address 127.6.40.65:52987)
0806 22:50:00.470849 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:00.470881 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:00.470963 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:00.471155 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:00.471156 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:00.471158 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:50:00.471253 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:00.471255 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:00.472310 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:00.472311 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:00.473657 (+ 1346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:00.473660 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:00.474518 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:00.474519 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:00.474616 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:00.474618 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 22:50:00.474681 (+ 63us) client_negotiation.cc:240] Negotiation successful
0806 22:50:00.474732 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49,"mutex_wait_us":6}
I20260806 22:50:00.474843 28640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:00.470918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.66:37655 (local address 127.6.40.65:35455)
0806 22:50:00.471142 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:00.471152 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:00.471210 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:00.471421 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:00.471422 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:00.471423 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:50:00.471478 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:00.471479 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:00.472424 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:00.472425 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:00.473657 (+ 1232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:00.473660 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:00.474525 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:00.474526 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:00.474611 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:00.474616 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:50:00.474681 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 22:50:00.474732 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"mutex_wait_us":14,"thread_start_us":138,"threads_started":1}
I20260806 22:50:00.475315 28356 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8da3881d0165434388d5f6fd4604a57f" candidate_uuid: "3a75f07037a54584a1b8d1b3bd13336a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a152c25bb71949a0a62dce1e0476c6d3" is_pre_election: true
I20260806 22:50:00.475310 28529 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8da3881d0165434388d5f6fd4604a57f" candidate_uuid: "3a75f07037a54584a1b8d1b3bd13336a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "baf570133f8749b68b951eaf0fcd9758" is_pre_election: true
I20260806 22:50:00.475488 28356 raft_consensus.cc:2454] T 8da3881d0165434388d5f6fd4604a57f P a152c25bb71949a0a62dce1e0476c6d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a75f07037a54584a1b8d1b3bd13336a in term 0.
I20260806 22:50:00.475503 28529 raft_consensus.cc:2454] T 8da3881d0165434388d5f6fd4604a57f P baf570133f8749b68b951eaf0fcd9758 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a75f07037a54584a1b8d1b3bd13336a in term 0.
I20260806 22:50:00.475749 28079 leader_election.cc:302] T 8da3881d0165434388d5f6fd4604a57f P 3a75f07037a54584a1b8d1b3bd13336a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a75f07037a54584a1b8d1b3bd13336a, baf570133f8749b68b951eaf0fcd9758; no voters:
I20260806 22:50:00.475957 28639 raft_consensus.cc:2781] T 8da3881d0165434388d5f6fd4604a57f P 3a75f07037a54584a1b8d1b3bd13336a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:50:00.476052 28639 raft_consensus.cc:482] T 8da3881d0165434388d5f6fd4604a57f P 3a75f07037a54584a1b8d1b3bd13336a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:50:00.476083 28639 raft_consensus.cc:3037] T 8da3881d0165434388d5f6fd4604a57f P 3a75f07037a54584a1b8d1b3bd13336a [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:50:00.477064 28639 raft_consensus.cc:504] T 8da3881d0165434388d5f6fd4604a57f P 3a75f07037a54584a1b8d1b3bd13336a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baf570133f8749b68b951eaf0fcd9758" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 32857 } } peers { permanent_uuid: "3a75f07037a54584a1b8d1b3bd13336a" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 37891 } } peers { permanent_uuid: "a152c25bb71949a0a62dce1e0476c6d3" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 37655 } }
I20260806 22:50:00.477205 28639 leader_election.cc:288] T 8da3881d0165434388d5f6fd4604a57f P 3a75f07037a54584a1b8d1b3bd13336a [CANDIDATE]: Term 1 election: Requested vote from peers baf570133f8749b68b951eaf0fcd9758 (127.6.40.67:32857), a152c25bb71949a0a62dce1e0476c6d3 (127.6.40.66:37655)
I20260806 22:50:00.477577 28529 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8da3881d0165434388d5f6fd4604a57f" candidate_uuid: "3a75f07037a54584a1b8d1b3bd13336a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "baf570133f8749b68b951eaf0fcd9758"
I20260806 22:50:00.477596 28356 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8da3881d0165434388d5f6fd4604a57f" candidate_uuid: "3a75f07037a54584a1b8d1b3bd13336a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a152c25bb71949a0a62dce1e0476c6d3"
I20260806 22:50:00.477677 28356 raft_consensus.cc:3037] T 8da3881d0165434388d5f6fd4604a57f P a152c25bb71949a0a62dce1e0476c6d3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:50:00.477676 28529 raft_consensus.cc:3037] T 8da3881d0165434388d5f6fd4604a57f P baf570133f8749b68b951eaf0fcd9758 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:50:00.478595 28529 raft_consensus.cc:2454] T 8da3881d0165434388d5f6fd4604a57f P baf570133f8749b68b951eaf0fcd9758 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a75f07037a54584a1b8d1b3bd13336a in term 1.
I20260806 22:50:00.478607 28356 raft_consensus.cc:2454] T 8da3881d0165434388d5f6fd4604a57f P a152c25bb71949a0a62dce1e0476c6d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a75f07037a54584a1b8d1b3bd13336a in term 1.
I20260806 22:50:00.478802 28079 leader_election.cc:302] T 8da3881d0165434388d5f6fd4604a57f P 3a75f07037a54584a1b8d1b3bd13336a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a75f07037a54584a1b8d1b3bd13336a, a152c25bb71949a0a62dce1e0476c6d3; no voters:
I20260806 22:50:00.478948 28639 raft_consensus.cc:2781] T 8da3881d0165434388d5f6fd4604a57f P 3a75f07037a54584a1b8d1b3bd13336a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:50:00.479161 28639 raft_consensus.cc:686] T 8da3881d0165434388d5f6fd4604a57f P 3a75f07037a54584a1b8d1b3bd13336a [term 1 LEADER]: Becoming Leader. State: Replica: 3a75f07037a54584a1b8d1b3bd13336a, State: Running, Role: LEADER
I20260806 22:50:00.479288 28639 consensus_queue.cc:237] T 8da3881d0165434388d5f6fd4604a57f P 3a75f07037a54584a1b8d1b3bd13336a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baf570133f8749b68b951eaf0fcd9758" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 32857 } } peers { permanent_uuid: "3a75f07037a54584a1b8d1b3bd13336a" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 37891 } } peers { permanent_uuid: "a152c25bb71949a0a62dce1e0476c6d3" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 37655 } }
I20260806 22:50:00.480430 27971 catalog_manager.cc:5495] T 8da3881d0165434388d5f6fd4604a57f P 3a75f07037a54584a1b8d1b3bd13336a reported cstate change: term changed from 0 to 1, leader changed from <none> to 3a75f07037a54584a1b8d1b3bd13336a (127.6.40.65). New cstate: current_term: 1 leader_uuid: "3a75f07037a54584a1b8d1b3bd13336a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baf570133f8749b68b951eaf0fcd9758" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 32857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a75f07037a54584a1b8d1b3bd13336a" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 37891 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a152c25bb71949a0a62dce1e0476c6d3" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 37655 } health_report { overall_health: UNKNOWN } } }
I20260806 22:50:00.489647 28619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:00.487103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54578 (local address 127.6.40.65:37891)
0806 22:50:00.487166 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:00.487169 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:00.487178 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:50:00.487221 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:00.487222 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:00.487239 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:00.487317 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:50:00.487476 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:00.487949 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:00.488743 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:00.488999 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:00.489075 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:00.489260 (+ 185us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:50:00.489373 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:50:00.489487 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:00.489507 (+ 20us) server_negotiation.cc:299] Negotiation successful
0806 22:50:00.489528 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 22:50:00.491675 28356 raft_consensus.cc:1261] T 8da3881d0165434388d5f6fd4604a57f P a152c25bb71949a0a62dce1e0476c6d3 [term 1 FOLLOWER]: Refusing update from remote peer 3a75f07037a54584a1b8d1b3bd13336a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:50:00.491675 28529 raft_consensus.cc:1261] T 8da3881d0165434388d5f6fd4604a57f P baf570133f8749b68b951eaf0fcd9758 [term 1 FOLLOWER]: Refusing update from remote peer 3a75f07037a54584a1b8d1b3bd13336a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:50:00.492134 28641 consensus_queue.cc:1035] T 8da3881d0165434388d5f6fd4604a57f P 3a75f07037a54584a1b8d1b3bd13336a [LEADER]: Connected to new peer: Peer: permanent_uuid: "baf570133f8749b68b951eaf0fcd9758" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 32857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:50:00.492332 28639 consensus_queue.cc:1035] T 8da3881d0165434388d5f6fd4604a57f P 3a75f07037a54584a1b8d1b3bd13336a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a152c25bb71949a0a62dce1e0476c6d3" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 37655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:50:00.496194 28646 mvcc.cc:205] Tried to move back new op lower bound from 7315687835610525696 to 7315687835564204032. Current Snapshot: MvccSnapshot[applied={T|T < 7315687835610525696 or (T in {7315687835610525696})}]
I20260806 22:50:00.497653 27970 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59162:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:50:00.497754 27970 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59162:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:50:00.498546 27970 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a7f7cb97a9a4b75b6c39701b1d3e39f: Sending DeleteTablet for 3 replicas of tablet 8da3881d0165434388d5f6fd4604a57f
I20260806 22:50:00.498875 28517 tablet_service.cc:1514] Processing DeleteTablet for tablet 8da3881d0165434388d5f6fd4604a57f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:50:00 UTC) from {username='kudu', principal='kudu/127.6.40.126@KRBTEST.COM'} at 127.0.0.1:41630
I20260806 22:50:00.498901 28147 tablet_service.cc:1514] Processing DeleteTablet for tablet 8da3881d0165434388d5f6fd4604a57f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:50:00 UTC) from {username='kudu', principal='kudu/127.6.40.126@KRBTEST.COM'} at 127.0.0.1:54562
I20260806 22:50:00.498898 28328 tablet_service.cc:1514] Processing DeleteTablet for tablet 8da3881d0165434388d5f6fd4604a57f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:50:00 UTC) from {username='kudu', principal='kudu/127.6.40.126@KRBTEST.COM'} at 127.0.0.1:47258
I20260806 22:50:00.499251 28665 tablet_replica.cc:330] T 8da3881d0165434388d5f6fd4604a57f P baf570133f8749b68b951eaf0fcd9758: stopping tablet replica
I20260806 22:50:00.499284 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 28054
I20260806 22:50:00.499372 28665 raft_consensus.cc:2229] T 8da3881d0165434388d5f6fd4604a57f P baf570133f8749b68b951eaf0fcd9758 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:50:00.499480 28665 raft_consensus.cc:2258] T 8da3881d0165434388d5f6fd4604a57f P baf570133f8749b68b951eaf0fcd9758 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:50:00.499626 28667 tablet_replica.cc:330] T 8da3881d0165434388d5f6fd4604a57f P a152c25bb71949a0a62dce1e0476c6d3: stopping tablet replica
I20260806 22:50:00.499712 28667 raft_consensus.cc:2229] T 8da3881d0165434388d5f6fd4604a57f P a152c25bb71949a0a62dce1e0476c6d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:50:00.499799 28667 raft_consensus.cc:2258] T 8da3881d0165434388d5f6fd4604a57f P a152c25bb71949a0a62dce1e0476c6d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:50:00.504549 28665 ts_tablet_manager.cc:1905] T 8da3881d0165434388d5f6fd4604a57f P baf570133f8749b68b951eaf0fcd9758: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:50:00.507115 28665 ts_tablet_manager.cc:1918] T 8da3881d0165434388d5f6fd4604a57f P baf570133f8749b68b951eaf0fcd9758: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:50:00.507232 28665 log.cc:1196] T 8da3881d0165434388d5f6fd4604a57f P baf570133f8749b68b951eaf0fcd9758: Deleting WAL directory at /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-2/wal/wals/8da3881d0165434388d5f6fd4604a57f
I20260806 22:50:00.507591 28665 ts_tablet_manager.cc:1939] T 8da3881d0165434388d5f6fd4604a57f P baf570133f8749b68b951eaf0fcd9758: Deleting consensus metadata
I20260806 22:50:00.508076 27953 catalog_manager.cc:4841] TS baf570133f8749b68b951eaf0fcd9758 (127.6.40.67:32857): tablet 8da3881d0165434388d5f6fd4604a57f (table test-table [id=e682718ffa3c4fe0b840ce13f3ac375f]) successfully deleted
I20260806 22:50:00.508822 28667 ts_tablet_manager.cc:1905] T 8da3881d0165434388d5f6fd4604a57f P a152c25bb71949a0a62dce1e0476c6d3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:50:00.511176 28667 ts_tablet_manager.cc:1918] T 8da3881d0165434388d5f6fd4604a57f P a152c25bb71949a0a62dce1e0476c6d3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:50:00.511325 28667 log.cc:1196] T 8da3881d0165434388d5f6fd4604a57f P a152c25bb71949a0a62dce1e0476c6d3: Deleting WAL directory at /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-1/wal/wals/8da3881d0165434388d5f6fd4604a57f
I20260806 22:50:00.511672 28667 ts_tablet_manager.cc:1939] T 8da3881d0165434388d5f6fd4604a57f P a152c25bb71949a0a62dce1e0476c6d3: Deleting consensus metadata
I20260806 22:50:00.512188 27953 catalog_manager.cc:4841] TS a152c25bb71949a0a62dce1e0476c6d3 (127.6.40.66:37655): tablet 8da3881d0165434388d5f6fd4604a57f (table test-table [id=e682718ffa3c4fe0b840ce13f3ac375f]) successfully deleted
W20260806 22:50:00.516458 27952 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.40.65:37891 (error 108)
I20260806 22:50:00.516692 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 28236
I20260806 22:50:00.521255 28615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:00.521104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.65:37891 (local address 127.0.0.1:54584)
0806 22:50:00.521161 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:00.521209 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.65:37891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260806 22:50:00.521402 27952 catalog_manager.cc:4584] TS 3a75f07037a54584a1b8d1b3bd13336a (127.6.40.65:37891): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8da3881d0165434388d5f6fd4604a57f: Network error: Client connection negotiation failed: client connection to 127.6.40.65:37891: connect: Connection refused (error 111)
I20260806 22:50:00.525696 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 28425
I20260806 22:50:00.533151 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 27919
I20260806 22:50:20.542659 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.126:42633
--webserver_interface=127.6.40.126
--webserver_port=43717
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab
--principal=kudu/127.6.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:36473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.40.126:42633
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:50:20.628731 28670 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:50:20.628993 28670 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:50:20.629031 28670 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:50:20.630172 28670 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 22:50:20.630236 28670 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 22:50:20.630259 28670 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 22:50:20.630278 28670 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:50:20.630309 28670 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:50:20.630340 28670 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:50:20.630367 28670 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:50:20.630409 28670 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:50:20.631877 28670 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:36473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.6.40.126:42633
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab
--principal=kudu/127.6.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.126:42633
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.40.126
--webserver_port=43717
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28670
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:50:20.632156 28670 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:50:20.632393 28670 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:50:20.642323 28682 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:20.642372 28679 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:20.642485 28670 server_base.cc:1034] running on GCE node
W20260806 22:50:20.642367 28678 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:20.642901 28670 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:50:20.643307 28670 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:50:20.644486 28670 hybrid_clock.cc:648] HybridClock initialized: now 1786056620644468 us; error 39 us; skew 500 ppm
Aug 06 22:50:20 dist-test-slave-zh2d krb5kdc[27902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056620, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:50:20.647404 28670 init.cc:377] Logged in from keytab as kudu/127.6.40.126@KRBTEST.COM (short username kudu)
I20260806 22:50:20.647862 28670 webserver.cc:460] Webserver started at http://127.6.40.126:43717/ using document root <none> and password file <none>
I20260806 22:50:20.648079 28670 fs_manager.cc:359] Metadata directory not provided
I20260806 22:50:20.648137 28670 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:50:20.649717 28670 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.000s
I20260806 22:50:20.650476 28692 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:20.650694 28670 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 22:50:20.650803 28670 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/master-0/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/master-0/wal
uuid: "0a7f7cb97a9a4b75b6c39701b1d3e39f"
format_stamp: "Formatted at 2026-08-06 22:49:59 on dist-test-slave-zh2d"
server_key: "b909af0f5c21d28eae37cac377290833"
server_key_iv: "a755d12d30bf58a14981a6b7044aff99"
server_key_version: "encryptionkey@0"
I20260806 22:50:20.651230 28670 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:50:20.671113 28670 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:50:20.673231 28670 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:50:20.673415 28670 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:50:20.678215 28670 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.126:42633
I20260806 22:50:20.678270 28763 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.126:42633 every 8 connection(s)
I20260806 22:50:20.678684 28670 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
I20260806 22:50:20.679198 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 28670
I20260806 22:50:20.679528 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.65:37891
--local_ip_for_outbound_sockets=127.6.40.65
--tserver_master_addrs=127.6.40.126:42633
--webserver_port=34111
--webserver_interface=127.6.40.65
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.65.keytab
--principal=kudu/127.6.40.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:36473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:50:20.680555 28764 sys_catalog.cc:263] Verifying existing consensus state
I20260806 22:50:20.681479 28764 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a7f7cb97a9a4b75b6c39701b1d3e39f: Bootstrap starting.
I20260806 22:50:20.691669 28764 log.cc:824] T 00000000000000000000000000000000 P 0a7f7cb97a9a4b75b6c39701b1d3e39f: Log is configured to *not* fsync() on all Append() calls
I20260806 22:50:20.694154 28764 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a7f7cb97a9a4b75b6c39701b1d3e39f: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 22:50:20.694564 28764 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a7f7cb97a9a4b75b6c39701b1d3e39f: Bootstrap complete.
I20260806 22:50:20.697317 28764 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0a7f7cb97a9a4b75b6c39701b1d3e39f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a7f7cb97a9a4b75b6c39701b1d3e39f" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 42633 } }
I20260806 22:50:20.697608 28764 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0a7f7cb97a9a4b75b6c39701b1d3e39f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a7f7cb97a9a4b75b6c39701b1d3e39f, State: Initialized, Role: FOLLOWER
I20260806 22:50:20.697855 28764 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0a7f7cb97a9a4b75b6c39701b1d3e39f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a7f7cb97a9a4b75b6c39701b1d3e39f" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 42633 } }
I20260806 22:50:20.697966 28764 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0a7f7cb97a9a4b75b6c39701b1d3e39f [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:50:20.698032 28764 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0a7f7cb97a9a4b75b6c39701b1d3e39f [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:50:20.698087 28764 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0a7f7cb97a9a4b75b6c39701b1d3e39f [term 1 FOLLOWER]: Advancing to term 2
I20260806 22:50:20.699237 28764 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0a7f7cb97a9a4b75b6c39701b1d3e39f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a7f7cb97a9a4b75b6c39701b1d3e39f" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 42633 } }
I20260806 22:50:20.699378 28764 leader_election.cc:302] T 00000000000000000000000000000000 P 0a7f7cb97a9a4b75b6c39701b1d3e39f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0a7f7cb97a9a4b75b6c39701b1d3e39f; no voters:
I20260806 22:50:20.699616 28764 leader_election.cc:288] T 00000000000000000000000000000000 P 0a7f7cb97a9a4b75b6c39701b1d3e39f [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 22:50:20.699657 28768 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0a7f7cb97a9a4b75b6c39701b1d3e39f [term 2 FOLLOWER]: Leader election won for term 2
I20260806 22:50:20.699795 28768 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0a7f7cb97a9a4b75b6c39701b1d3e39f [term 2 LEADER]: Becoming Leader. State: Replica: 0a7f7cb97a9a4b75b6c39701b1d3e39f, State: Running, Role: LEADER
I20260806 22:50:20.699896 28768 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0a7f7cb97a9a4b75b6c39701b1d3e39f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a7f7cb97a9a4b75b6c39701b1d3e39f" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 42633 } }
I20260806 22:50:20.699942 28764 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0a7f7cb97a9a4b75b6c39701b1d3e39f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:50:20.700172 28768 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a7f7cb97a9a4b75b6c39701b1d3e39f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0a7f7cb97a9a4b75b6c39701b1d3e39f. Latest consensus state: current_term: 2 leader_uuid: "0a7f7cb97a9a4b75b6c39701b1d3e39f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a7f7cb97a9a4b75b6c39701b1d3e39f" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 42633 } } }
I20260806 22:50:20.700258 28768 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a7f7cb97a9a4b75b6c39701b1d3e39f [sys.catalog]: This master's current role is: LEADER
I20260806 22:50:20.700178 28769 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a7f7cb97a9a4b75b6c39701b1d3e39f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "0a7f7cb97a9a4b75b6c39701b1d3e39f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a7f7cb97a9a4b75b6c39701b1d3e39f" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 42633 } } }
I20260806 22:50:20.700453 28769 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a7f7cb97a9a4b75b6c39701b1d3e39f [sys.catalog]: This master's current role is: LEADER
I20260806 22:50:20.700496 28774 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:50:20.701507 28774 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:50:20.701663 28774 catalog_manager.cc:1259] Loaded cluster ID: b6dca6ddc3014539b3d05e506340117b
I20260806 22:50:20.701890 28774 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:50:20.702857 28774 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:50:20.703083 28774 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 0a7f7cb97a9a4b75b6c39701b1d3e39f: Loaded TSK: 0
I20260806 22:50:20.714000 28774 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0a7f7cb97a9a4b75b6c39701b1d3e39f: Generated new TSK 1
I20260806 22:50:20.714121 28774 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 22:50:20.786420 28766 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:50:20.786654 28766 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:50:20.786695 28766 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:50:20.787837 28766 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:50:20.787905 28766 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:50:20.787952 28766 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.65
I20260806 22:50:20.789517 28766 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:36473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.65.keytab
--principal=kudu/127.6.40.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.65:37891
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.40.65
--webserver_port=34111
--tserver_master_addrs=127.6.40.126:42633
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28766
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.65
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:50:20.789780 28766 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:50:20.790009 28766 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:50:20.795154 28803 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:20.795267 28804 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:20.795441 28766 server_base.cc:1034] running on GCE node
W20260806 22:50:20.795526 28806 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:20.795744 28766 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:50:20.796028 28766 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:50:20.797192 28766 hybrid_clock.cc:648] HybridClock initialized: now 1786056620797163 us; error 46 us; skew 500 ppm
Aug 06 22:50:20 dist-test-slave-zh2d krb5kdc[27902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056620, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:50:20.799897 28766 init.cc:377] Logged in from keytab as kudu/127.6.40.65@KRBTEST.COM (short username kudu)
I20260806 22:50:20.800480 28766 webserver.cc:460] Webserver started at http://127.6.40.65:34111/ using document root <none> and password file <none>
I20260806 22:50:20.800705 28766 fs_manager.cc:359] Metadata directory not provided
I20260806 22:50:20.800792 28766 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:50:20.802349 28766 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:50:20.803100 28814 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:20.803320 28766 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:50:20.803436 28766 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-0/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-0/wal
uuid: "3a75f07037a54584a1b8d1b3bd13336a"
format_stamp: "Formatted at 2026-08-06 22:50:00 on dist-test-slave-zh2d"
server_key: "25aba12f593f79340ee874c9a3eb7765"
server_key_iv: "e23616f1a04dbb5dbf1f790b1201bbde"
server_key_version: "encryptionkey@0"
I20260806 22:50:20.803783 28766 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:50:20.821482 28766 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:50:20.823721 28766 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:50:20.823895 28766 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:50:20.824142 28766 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:50:20.824630 28828 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 22:50:20.826594 28766 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 22:50:20.826685 28766 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260806 22:50:20.826762 28766 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 22:50:20.827651 28766 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 22:50:20.827710 28766 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260806 22:50:20.827769 28828 tablet_bootstrap.cc:492] T 8da3881d0165434388d5f6fd4604a57f P 3a75f07037a54584a1b8d1b3bd13336a: Bootstrap starting.
I20260806 22:50:20.835438 28766 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.65:37891
I20260806 22:50:20.835819 28766 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
I20260806 22:50:20.837052 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 28766
I20260806 22:50:20.837360 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.66:37655
--local_ip_for_outbound_sockets=127.6.40.66
--tserver_master_addrs=127.6.40.126:42633
--webserver_port=33437
--webserver_interface=127.6.40.66
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.66.keytab
--principal=kudu/127.6.40.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:36473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:50:20.841861 28828 log.cc:824] T 8da3881d0165434388d5f6fd4604a57f P 3a75f07037a54584a1b8d1b3bd13336a: Log is configured to *not* fsync() on all Append() calls
I20260806 22:50:20.842777 28988 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.65:37891 every 8 connection(s)
I20260806 22:50:20.844316 28828 tablet_bootstrap.cc:492] T 8da3881d0165434388d5f6fd4604a57f P 3a75f07037a54584a1b8d1b3bd13336a: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 22:50:20.844624 28828 tablet_bootstrap.cc:492] T 8da3881d0165434388d5f6fd4604a57f P 3a75f07037a54584a1b8d1b3bd13336a: Bootstrap complete.
I20260806 22:50:20.846112 28828 ts_tablet_manager.cc:1397] T 8da3881d0165434388d5f6fd4604a57f P 3a75f07037a54584a1b8d1b3bd13336a: Time spent bootstrapping tablet: real 0.018s user 0.005s sys 0.010s
I20260806 22:50:20.847886 28828 raft_consensus.cc:348] T 8da3881d0165434388d5f6fd4604a57f P 3a75f07037a54584a1b8d1b3bd13336a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baf570133f8749b68b951eaf0fcd9758" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 32857 } } peers { permanent_uuid: "3a75f07037a54584a1b8d1b3bd13336a" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 37891 } } peers { permanent_uuid: "a152c25bb71949a0a62dce1e0476c6d3" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 37655 } }
I20260806 22:50:20.848340 28828 raft_consensus.cc:729] T 8da3881d0165434388d5f6fd4604a57f P 3a75f07037a54584a1b8d1b3bd13336a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a75f07037a54584a1b8d1b3bd13336a, State: Initialized, Role: FOLLOWER
I20260806 22:50:20.848474 28828 consensus_queue.cc:260] T 8da3881d0165434388d5f6fd4604a57f P 3a75f07037a54584a1b8d1b3bd13336a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baf570133f8749b68b951eaf0fcd9758" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 32857 } } peers { permanent_uuid: "3a75f07037a54584a1b8d1b3bd13336a" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 37891 } } peers { permanent_uuid: "a152c25bb71949a0a62dce1e0476c6d3" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 37655 } }
I20260806 22:50:20.848783 28828 ts_tablet_manager.cc:1428] T 8da3881d0165434388d5f6fd4604a57f P 3a75f07037a54584a1b8d1b3bd13336a: Time spent starting tablet: real 0.003s user 0.001s sys 0.003s
Aug 06 22:50:20 dist-test-slave-zh2d krb5kdc[27902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056620, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.65@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
I20260806 22:50:20.853848 28993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:20.836596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:42633 (local address 127.6.40.65:53817)
0806 22:50:20.836810 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:20.836887 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:20.837589 (+ 702us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:20.842655 (+ 5066us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:20.842659 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:20.843110 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:20.843368 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:20.843386 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:20.845163 (+ 1777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:20.845164 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:20.846433 (+ 1269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:20.846438 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:20.847377 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:20.847378 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:20.847516 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:20.848164 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:20.848174 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:20.849923 (+ 1749us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:50:20.852107 (+ 2184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:20.852115 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:20.852117 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:20.852390 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:20.852644 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:20.852645 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:20.852645 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:20.852763 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:20.853244 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:50:20.853251 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:50:20.853390 (+ 139us) client_negotiation.cc:769] Sending connection context
0806 22:50:20.853485 (+ 95us) client_negotiation.cc:240] Negotiation successful
0806 22:50:20.853644 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":136,"threads_started":1}
I20260806 22:50:20.854542 28994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:20.836736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.65:53817 (local address 127.6.40.126:42633)
0806 22:50:20.842158 (+ 5422us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:20.842168 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:20.842215 (+ 47us) server_negotiation.cc:407] Connection header received
0806 22:50:20.842360 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:20.842369 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:20.842439 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:20.842607 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:20.843533 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:20.845008 (+ 1475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:20.846540 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:20.847154 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:20.847259 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:20.850036 (+ 2777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:20.850040 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:20.850040 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:20.850053 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:20.852015 (+ 1962us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:20.852473 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:20.852476 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:20.852476 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:20.852579 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:20.852825 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:20.852826 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:20.852827 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:20.853140 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:20.853215 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:20.854180 (+ 965us) server_negotiation.cc:299] Negotiation successful
0806 22:50:20.854354 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5366,"thread_start_us":117,"threads_started":1}
I20260806 22:50:20.855063 28989 heartbeater.cc:347] Connected to a master server at 127.6.40.126:42633
I20260806 22:50:20.855158 28989 heartbeater.cc:464] Registering TS with master...
I20260806 22:50:20.855350 28989 heartbeater.cc:511] Master 127.6.40.126:42633 requested a full tablet report, sending...
I20260806 22:50:20.856060 28713 ts_manager.cc:194] Registered new tserver with Master: 3a75f07037a54584a1b8d1b3bd13336a (127.6.40.65:37891)
I20260806 22:50:20.857271 28713 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.40.65@KRBTEST.COM'} at 127.6.40.65:53817
I20260806 22:50:20.858176 28989 heartbeater.cc:503] Master 127.6.40.126:42633 was elected leader, sending a full tablet report...
I20260806 22:50:20.862259 28999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:20.857078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.65:37891 (local address 127.0.0.1:53020)
0806 22:50:20.857557 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:20.857593 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:20.857657 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:20.858190 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:20.858194 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:20.858197 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:50:20.858372 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:20.858374 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:20.859672 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:20.859673 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:20.861006 (+ 1333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:20.861009 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:20.861911 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:20.861912 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:20.862026 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:20.862042 (+ 16us) client_negotiation.cc:769] Sending connection context
0806 22:50:20.862132 (+ 90us) client_negotiation.cc:240] Negotiation successful
0806 22:50:20.862194 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":86,"threads_started":1}
I20260806 22:50:20.862262 29000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:20.857344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53020 (local address 127.6.40.65:37891)
0806 22:50:20.857783 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:20.857788 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:20.857797 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:50:20.857828 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:20.857830 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:20.858043 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:20.858463 (+ 420us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:50:20.858509 (+ 46us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:20.859570 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:20.861104 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:20.861806 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:20.861889 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:20.861938 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:20.862154 (+ 216us) server_negotiation.cc:299] Negotiation successful
0806 22:50:20.862195 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":411,"thread_start_us":68,"threads_started":1}
I20260806 22:50:20.862916 28887 tablet_service.cc:1514] Processing DeleteTablet for tablet 8da3881d0165434388d5f6fd4604a57f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:50:00 UTC) from {username='kudu', principal='kudu/127.6.40.126@KRBTEST.COM'} at 127.0.0.1:53020
I20260806 22:50:20.863514 29003 tablet_replica.cc:330] stopping tablet replica
I20260806 22:50:20.863618 29003 raft_consensus.cc:2229] T 8da3881d0165434388d5f6fd4604a57f P 3a75f07037a54584a1b8d1b3bd13336a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:50:20.863679 29003 raft_consensus.cc:2258] T 8da3881d0165434388d5f6fd4604a57f P 3a75f07037a54584a1b8d1b3bd13336a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:50:20.863984 29003 ts_tablet_manager.cc:1905] T 8da3881d0165434388d5f6fd4604a57f P 3a75f07037a54584a1b8d1b3bd13336a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:50:20.866106 29003 ts_tablet_manager.cc:1918] T 8da3881d0165434388d5f6fd4604a57f P 3a75f07037a54584a1b8d1b3bd13336a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:50:20.866178 29003 log.cc:1196] T 8da3881d0165434388d5f6fd4604a57f P 3a75f07037a54584a1b8d1b3bd13336a: Deleting WAL directory at /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-0/wal/wals/8da3881d0165434388d5f6fd4604a57f
I20260806 22:50:20.866430 29003 ts_tablet_manager.cc:1939] T 8da3881d0165434388d5f6fd4604a57f P 3a75f07037a54584a1b8d1b3bd13336a: Deleting consensus metadata
I20260806 22:50:20.866930 28695 catalog_manager.cc:4841] TS 3a75f07037a54584a1b8d1b3bd13336a (127.6.40.65:37891): tablet 8da3881d0165434388d5f6fd4604a57f (table test-table [id=e682718ffa3c4fe0b840ce13f3ac375f]) successfully deleted
W20260806 22:50:20.932574 28996 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:50:20.932811 28996 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:50:20.932847 28996 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:50:20.934033 28996 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:50:20.934099 28996 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:50:20.934142 28996 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.66
I20260806 22:50:20.935673 28996 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:36473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.66.keytab
--principal=kudu/127.6.40.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.66:37655
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.40.66
--webserver_port=33437
--tserver_master_addrs=127.6.40.126:42633
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28996
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.66
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:50:20.935935 28996 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:50:20.936177 28996 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:50:20.941525 29009 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:20.941681 28996 server_base.cc:1034] running on GCE node
W20260806 22:50:20.941534 29011 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:20.941540 29015 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:20.941990 28996 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:50:20.942260 28996 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:50:20.943442 28996 hybrid_clock.cc:648] HybridClock initialized: now 1786056620943448 us; error 62 us; skew 500 ppm
Aug 06 22:50:20 dist-test-slave-zh2d krb5kdc[27902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056620, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:50:20.946081 28996 init.cc:377] Logged in from keytab as kudu/127.6.40.66@KRBTEST.COM (short username kudu)
I20260806 22:50:20.946539 28996 webserver.cc:460] Webserver started at http://127.6.40.66:33437/ using document root <none> and password file <none>
I20260806 22:50:20.946769 28996 fs_manager.cc:359] Metadata directory not provided
I20260806 22:50:20.946851 28996 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:50:20.948215 28996 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:50:20.948943 29024 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:20.949180 28996 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:50:20.949368 28996 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-1/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-1/wal
uuid: "a152c25bb71949a0a62dce1e0476c6d3"
format_stamp: "Formatted at 2026-08-06 22:50:00 on dist-test-slave-zh2d"
server_key: "9b0dcf3af3447da2fadf70f640bed9a7"
server_key_iv: "bd8b2fbc787a5c9153913649d60b0be1"
server_key_version: "encryptionkey@0"
I20260806 22:50:20.949772 28996 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:50:20.975135 28996 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:50:20.977591 28996 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:50:20.977804 28996 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:50:20.978106 28996 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:50:20.978541 28996 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:50:20.978610 28996 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:20.978665 28996 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:50:20.978696 28996 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:20.986239 28996 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.66:37655
I20260806 22:50:20.986325 29187 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.66:37655 every 8 connection(s)
I20260806 22:50:20.986742 28996 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
Aug 06 22:50:20 dist-test-slave-zh2d krb5kdc[27902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056620, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.66@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
I20260806 22:50:20.995285 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 28996
I20260806 22:50:20.995637 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.67:32857
--local_ip_for_outbound_sockets=127.6.40.67
--tserver_master_addrs=127.6.40.126:42633
--webserver_port=46549
--webserver_interface=127.6.40.67
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.67.keytab
--principal=kudu/127.6.40.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:36473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:50:20.996984 28994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:20.987596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.66:43309 (local address 127.6.40.126:42633)
0806 22:50:20.987672 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:20.987675 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:20.987947 (+ 272us) server_negotiation.cc:407] Connection header received
0806 22:50:20.988525 (+ 578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:20.988527 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:20.988542 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:20.988600 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:20.989788 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:20.990294 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:20.991312 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:20.991632 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:20.991690 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:20.993958 (+ 2268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:20.993959 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:20.993960 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:20.993966 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:20.995208 (+ 1242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:20.995769 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:20.995771 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:20.995772 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:20.995863 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:20.996184 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:20.996185 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:20.996185 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:20.996305 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:20.996383 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:20.996764 (+ 381us) server_negotiation.cc:299] Negotiation successful
0806 22:50:20.996917 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 22:50:20.997215 29191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:20.987574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:42633 (local address 127.6.40.66:43309)
0806 22:50:20.987809 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:20.987853 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:20.988421 (+ 568us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:20.988678 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:20.988682 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:20.989460 (+ 778us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:20.989682 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:20.989688 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:20.990414 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:20.990415 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:20.991219 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:20.991222 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:20.991723 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:20.991724 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:20.991841 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:20.992343 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:20.992352 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:20.993849 (+ 1497us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:50:20.995313 (+ 1464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:20.995321 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:20.995324 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:20.995675 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:20.995943 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:20.995944 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:20.995944 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:20.996092 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:20.996389 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:50:20.996396 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:50:20.996597 (+ 201us) client_negotiation.cc:769] Sending connection context
0806 22:50:20.996712 (+ 115us) client_negotiation.cc:240] Negotiation successful
0806 22:50:20.996921 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":108,"threads_started":1}
I20260806 22:50:20.997673 29188 heartbeater.cc:347] Connected to a master server at 127.6.40.126:42633
I20260806 22:50:20.997793 29188 heartbeater.cc:464] Registering TS with master...
I20260806 22:50:20.997982 29188 heartbeater.cc:511] Master 127.6.40.126:42633 requested a full tablet report, sending...
I20260806 22:50:20.998392 28713 ts_manager.cc:194] Registered new tserver with Master: a152c25bb71949a0a62dce1e0476c6d3 (127.6.40.66:37655)
I20260806 22:50:20.998771 28713 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.40.66@KRBTEST.COM'} at 127.6.40.66:43309
W20260806 22:50:21.081887 29195 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:50:21.082087 29195 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:50:21.082110 29195 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:50:21.083178 29195 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:50:21.083211 29195 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:50:21.083235 29195 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.67
I20260806 22:50:21.084787 29195 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:36473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.67.keytab
--principal=kudu/127.6.40.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.67:32857
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.40.67
--webserver_port=46549
--tserver_master_addrs=127.6.40.126:42633
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29195
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.67
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:50:21.085034 29195 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:50:21.085215 29195 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:50:21.090351 29201 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:21.090458 29203 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:21.090471 29206 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:21.090665 29195 server_base.cc:1034] running on GCE node
I20260806 22:50:21.090831 29195 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:50:21.091073 29195 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:50:21.092209 29195 hybrid_clock.cc:648] HybridClock initialized: now 1786056621092196 us; error 30 us; skew 500 ppm
Aug 06 22:50:21 dist-test-slave-zh2d krb5kdc[27902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056621, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:50:21.094810 29195 init.cc:377] Logged in from keytab as kudu/127.6.40.67@KRBTEST.COM (short username kudu)
I20260806 22:50:21.095239 29195 webserver.cc:460] Webserver started at http://127.6.40.67:46549/ using document root <none> and password file <none>
I20260806 22:50:21.095463 29195 fs_manager.cc:359] Metadata directory not provided
I20260806 22:50:21.095546 29195 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:50:21.096937 29195 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:50:21.097584 29214 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:21.097769 29195 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:50:21.097870 29195 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-2/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-2/wal
uuid: "baf570133f8749b68b951eaf0fcd9758"
format_stamp: "Formatted at 2026-08-06 22:50:00 on dist-test-slave-zh2d"
server_key: "65b9da73df9edc56555e857cec9777c2"
server_key_iv: "91a59b4ea6bcf5c50a1dd0b7036f0620"
server_key_version: "encryptionkey@0"
I20260806 22:50:21.098229 29195 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:50:21.123989 29195 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:50:21.126161 29195 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:50:21.126332 29195 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:50:21.126602 29195 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:50:21.126927 29195 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:50:21.126982 29195 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:21.127024 29195 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:50:21.127051 29195 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:21.133361 29195 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.67:32857
I20260806 22:50:21.133435 29377 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.67:32857 every 8 connection(s)
I20260806 22:50:21.133746 29195 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
Aug 06 22:50:21 dist-test-slave-zh2d krb5kdc[27902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056621, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.67@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
I20260806 22:50:21.142736 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 29195
I20260806 22:50:21.143395 28994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:21.134539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.67:42017 (local address 127.6.40.126:42633)
0806 22:50:21.134652 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:21.134654 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:21.134783 (+ 129us) server_negotiation.cc:407] Connection header received
0806 22:50:21.135277 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:21.135278 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:21.135288 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:21.135383 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:21.136295 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:21.136803 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:21.137899 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:21.138133 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:21.138204 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:21.140414 (+ 2210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:21.140415 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:21.140416 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:21.140423 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:21.141882 (+ 1459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:21.142244 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:21.142245 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:21.142245 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:21.142297 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:21.142578 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:21.142579 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:21.142579 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:21.142679 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:21.142730 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:21.143096 (+ 366us) server_negotiation.cc:299] Negotiation successful
0806 22:50:21.143309 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260806 22:50:21.143478 29380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:21.134464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:42633 (local address 127.6.40.67:42017)
0806 22:50:21.134635 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:21.134679 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:21.135188 (+ 509us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:21.135442 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:21.135445 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:21.136022 (+ 577us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:21.136211 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:21.136216 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:21.136948 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:21.136949 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:21.137779 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:21.137781 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:21.138218 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:21.138218 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:21.138318 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:21.138825 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:21.138833 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:21.140292 (+ 1459us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:50:21.141956 (+ 1664us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:21.141963 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:21.141965 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:21.142178 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:21.142355 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:21.142356 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:21.142356 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:21.142481 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:21.142745 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:50:21.142753 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:50:21.142924 (+ 171us) client_negotiation.cc:769] Sending connection context
0806 22:50:21.143056 (+ 132us) client_negotiation.cc:240] Negotiation successful
0806 22:50:21.143233 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":93,"threads_started":1}
I20260806 22:50:21.144105 29378 heartbeater.cc:347] Connected to a master server at 127.6.40.126:42633
I20260806 22:50:21.144197 29378 heartbeater.cc:464] Registering TS with master...
I20260806 22:50:21.144399 29378 heartbeater.cc:511] Master 127.6.40.126:42633 requested a full tablet report, sending...
I20260806 22:50:21.144870 28713 ts_manager.cc:194] Registered new tserver with Master: baf570133f8749b68b951eaf0fcd9758 (127.6.40.67:32857)
I20260806 22:50:21.145285 28713 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.40.67@KRBTEST.COM'} at 127.6.40.67:42017
I20260806 22:50:21.149646 29383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:21.143139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52634 (local address 127.6.40.126:42633)
0806 22:50:21.143481 (+ 342us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:21.143483 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:21.143492 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:50:21.143517 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:21.143518 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:21.143532 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:21.143601 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:21.144347 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:21.145023 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:21.145957 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:21.146215 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:21.146274 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:21.147453 (+ 1179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:21.147454 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:21.147454 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:21.147461 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:21.148645 (+ 1184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:21.149017 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:21.149017 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:21.149018 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:21.149072 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:21.149209 (+ 137us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:21.149209 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:21.149209 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:21.149294 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:21.149419 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:21.149455 (+ 36us) server_negotiation.cc:299] Negotiation successful
0806 22:50:21.149518 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":73,"threads_started":1}
I20260806 22:50:21.150442 6305 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:50:21.153264 29383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:21.150831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52638 (local address 127.6.40.126:42633)
0806 22:50:21.150869 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:21.150871 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:21.150979 (+ 108us) server_negotiation.cc:407] Connection header received
0806 22:50:21.151025 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:21.151026 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:21.151053 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:21.151110 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:50:21.151183 (+ 73us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:21.151659 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:21.152449 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:21.152646 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:21.152707 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:21.152874 (+ 167us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:50:21.152926 (+ 52us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0806 22:50:21.153201 (+ 275us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:52638: token expired
Metrics: {"server-negotiator.queue_time_us":16}
W20260806 22:50:21.153344 29383 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:52638: token expired
W20260806 22:50:21.153340 28608 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.6.40.126:42633)
I20260806 22:50:21.159130 29383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:21.153590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52648 (local address 127.6.40.126:42633)
0806 22:50:21.153632 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:21.153634 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:21.153677 (+ 43us) server_negotiation.cc:407] Connection header received
0806 22:50:21.153723 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:21.153724 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:21.153730 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:21.153772 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:21.154311 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:21.154756 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:21.155503 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:21.155682 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:21.155731 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:21.156738 (+ 1007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:21.156739 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:21.156739 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:21.156745 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:21.158219 (+ 1474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:21.158518 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:21.158519 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:21.158519 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:21.158566 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:21.158714 (+ 148us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:21.158714 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:21.158714 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:21.158821 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:21.158888 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:21.158978 (+ 90us) server_negotiation.cc:299] Negotiation successful
0806 22:50:21.159034 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 22:50:21.160279 28713 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52648:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:50:21.162184 28713 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:50:21.164569 28887 tablet_service.cc:1467] Processing CreateTablet for tablet 8124b7f396634d2995d5ff0133546ace (DEFAULT_TABLE table=test-table [id=92621089f0604836a15b7e1fd2037111]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:50:21.164784 28887 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8124b7f396634d2995d5ff0133546ace. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:50:21.166328 28828 tablet_bootstrap.cc:492] T 8124b7f396634d2995d5ff0133546ace P 3a75f07037a54584a1b8d1b3bd13336a: Bootstrap starting.
I20260806 22:50:21.166674 28828 tablet_bootstrap.cc:654] T 8124b7f396634d2995d5ff0133546ace P 3a75f07037a54584a1b8d1b3bd13336a: Neither blocks nor log segments found. Creating new log.
I20260806 22:50:21.167335 28828 tablet_bootstrap.cc:492] T 8124b7f396634d2995d5ff0133546ace P 3a75f07037a54584a1b8d1b3bd13336a: No bootstrap required, opened a new log
I20260806 22:50:21.167395 28828 ts_tablet_manager.cc:1397] T 8124b7f396634d2995d5ff0133546ace P 3a75f07037a54584a1b8d1b3bd13336a: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 22:50:21.167522 28828 raft_consensus.cc:348] T 8124b7f396634d2995d5ff0133546ace P 3a75f07037a54584a1b8d1b3bd13336a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a152c25bb71949a0a62dce1e0476c6d3" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 37655 } } peers { permanent_uuid: "baf570133f8749b68b951eaf0fcd9758" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 32857 } } peers { permanent_uuid: "3a75f07037a54584a1b8d1b3bd13336a" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 37891 } }
I20260806 22:50:21.167605 28828 raft_consensus.cc:374] T 8124b7f396634d2995d5ff0133546ace P 3a75f07037a54584a1b8d1b3bd13336a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:50:21.167665 28828 raft_consensus.cc:729] T 8124b7f396634d2995d5ff0133546ace P 3a75f07037a54584a1b8d1b3bd13336a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a75f07037a54584a1b8d1b3bd13336a, State: Initialized, Role: FOLLOWER
I20260806 22:50:21.167722 28828 consensus_queue.cc:260] T 8124b7f396634d2995d5ff0133546ace P 3a75f07037a54584a1b8d1b3bd13336a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a152c25bb71949a0a62dce1e0476c6d3" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 37655 } } peers { permanent_uuid: "baf570133f8749b68b951eaf0fcd9758" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 32857 } } peers { permanent_uuid: "3a75f07037a54584a1b8d1b3bd13336a" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 37891 } }
I20260806 22:50:21.167834 28828 ts_tablet_manager.cc:1428] T 8124b7f396634d2995d5ff0133546ace P 3a75f07037a54584a1b8d1b3bd13336a: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 22:50:21.167960 28999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:21.164103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.66:37655 (local address 127.0.0.1:33622)
0806 22:50:21.164142 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:21.164153 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:21.164190 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:21.164643 (+ 453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:21.164644 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:21.164646 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:50:21.164690 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:21.164691 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:21.165700 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:21.165701 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:21.166732 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:21.166734 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:21.167712 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:21.167713 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:21.167779 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:21.167784 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:50:21.167819 (+ 35us) client_negotiation.cc:240] Negotiation successful
0806 22:50:21.167862 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21}
I20260806 22:50:21.168164 29385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:21.164194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.67:32857 (local address 127.0.0.1:52552)
0806 22:50:21.164302 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:21.164311 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:21.164350 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:21.164773 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:21.164774 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:21.164776 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:50:21.164821 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:21.164822 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:21.165989 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:21.165990 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:21.167039 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:21.167042 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:21.167915 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:21.167916 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:21.167984 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:21.167987 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:50:21.168044 (+ 57us) client_negotiation.cc:240] Negotiation successful
0806 22:50:21.168090 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":74,"threads_started":1}
I20260806 22:50:21.168401 29389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:21.164421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52552 (local address 127.6.40.67:32857)
0806 22:50:21.164626 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:21.164631 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:21.164642 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:50:21.164681 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:21.164683 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:21.164698 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:21.164776 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:50:21.164905 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:21.165899 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:21.167127 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:21.167776 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:21.167889 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:21.167953 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:21.168272 (+ 319us) server_negotiation.cc:299] Negotiation successful
0806 22:50:21.168312 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":101,"threads_started":1}
I20260806 22:50:21.168438 29386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:21.164213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33622 (local address 127.6.40.66:37655)
0806 22:50:21.164436 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:21.164443 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:21.164454 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:50:21.164503 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:21.164505 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:21.164521 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:21.164639 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:50:21.164751 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:21.165601 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:21.166832 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:21.167602 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:21.168190 (+ 588us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:21.168265 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:21.168293 (+ 28us) server_negotiation.cc:299] Negotiation successful
0806 22:50:21.168332 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":101,"threads_started":1}
I20260806 22:50:21.169129 29283 tablet_service.cc:1467] Processing CreateTablet for tablet 8124b7f396634d2995d5ff0133546ace (DEFAULT_TABLE table=test-table [id=92621089f0604836a15b7e1fd2037111]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:50:21.169301 29094 tablet_service.cc:1467] Processing CreateTablet for tablet 8124b7f396634d2995d5ff0133546ace (DEFAULT_TABLE table=test-table [id=92621089f0604836a15b7e1fd2037111]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:50:21.169517 29283 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8124b7f396634d2995d5ff0133546ace. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:50:21.169615 29094 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8124b7f396634d2995d5ff0133546ace. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:50:21.173203 29391 tablet_bootstrap.cc:492] T 8124b7f396634d2995d5ff0133546ace P baf570133f8749b68b951eaf0fcd9758: Bootstrap starting.
I20260806 22:50:21.173835 29391 tablet_bootstrap.cc:654] T 8124b7f396634d2995d5ff0133546ace P baf570133f8749b68b951eaf0fcd9758: Neither blocks nor log segments found. Creating new log.
I20260806 22:50:21.174424 29391 log.cc:824] T 8124b7f396634d2995d5ff0133546ace P baf570133f8749b68b951eaf0fcd9758: Log is configured to *not* fsync() on all Append() calls
I20260806 22:50:21.174749 29392 tablet_bootstrap.cc:492] T 8124b7f396634d2995d5ff0133546ace P a152c25bb71949a0a62dce1e0476c6d3: Bootstrap starting.
I20260806 22:50:21.175266 29391 tablet_bootstrap.cc:492] T 8124b7f396634d2995d5ff0133546ace P baf570133f8749b68b951eaf0fcd9758: No bootstrap required, opened a new log
I20260806 22:50:21.175344 29391 ts_tablet_manager.cc:1397] T 8124b7f396634d2995d5ff0133546ace P baf570133f8749b68b951eaf0fcd9758: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:50:21.175496 29392 tablet_bootstrap.cc:654] T 8124b7f396634d2995d5ff0133546ace P a152c25bb71949a0a62dce1e0476c6d3: Neither blocks nor log segments found. Creating new log.
I20260806 22:50:21.176146 29392 log.cc:824] T 8124b7f396634d2995d5ff0133546ace P a152c25bb71949a0a62dce1e0476c6d3: Log is configured to *not* fsync() on all Append() calls
I20260806 22:50:21.176965 29392 tablet_bootstrap.cc:492] T 8124b7f396634d2995d5ff0133546ace P a152c25bb71949a0a62dce1e0476c6d3: No bootstrap required, opened a new log
I20260806 22:50:21.177040 29392 ts_tablet_manager.cc:1397] T 8124b7f396634d2995d5ff0133546ace P a152c25bb71949a0a62dce1e0476c6d3: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:50:21.177578 29391 raft_consensus.cc:348] T 8124b7f396634d2995d5ff0133546ace P baf570133f8749b68b951eaf0fcd9758 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a152c25bb71949a0a62dce1e0476c6d3" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 37655 } } peers { permanent_uuid: "baf570133f8749b68b951eaf0fcd9758" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 32857 } } peers { permanent_uuid: "3a75f07037a54584a1b8d1b3bd13336a" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 37891 } }
I20260806 22:50:21.177760 29391 raft_consensus.cc:374] T 8124b7f396634d2995d5ff0133546ace P baf570133f8749b68b951eaf0fcd9758 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:50:21.177798 29391 raft_consensus.cc:729] T 8124b7f396634d2995d5ff0133546ace P baf570133f8749b68b951eaf0fcd9758 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: baf570133f8749b68b951eaf0fcd9758, State: Initialized, Role: FOLLOWER
I20260806 22:50:21.177937 29391 consensus_queue.cc:260] T 8124b7f396634d2995d5ff0133546ace P baf570133f8749b68b951eaf0fcd9758 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a152c25bb71949a0a62dce1e0476c6d3" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 37655 } } peers { permanent_uuid: "baf570133f8749b68b951eaf0fcd9758" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 32857 } } peers { permanent_uuid: "3a75f07037a54584a1b8d1b3bd13336a" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 37891 } }
I20260806 22:50:21.178224 29391 ts_tablet_manager.cc:1428] T 8124b7f396634d2995d5ff0133546ace P baf570133f8749b68b951eaf0fcd9758: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:50:21.178256 29378 heartbeater.cc:503] Master 127.6.40.126:42633 was elected leader, sending a full tablet report...
I20260806 22:50:21.179073 29392 raft_consensus.cc:348] T 8124b7f396634d2995d5ff0133546ace P a152c25bb71949a0a62dce1e0476c6d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a152c25bb71949a0a62dce1e0476c6d3" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 37655 } } peers { permanent_uuid: "baf570133f8749b68b951eaf0fcd9758" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 32857 } } peers { permanent_uuid: "3a75f07037a54584a1b8d1b3bd13336a" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 37891 } }
I20260806 22:50:21.179219 29392 raft_consensus.cc:374] T 8124b7f396634d2995d5ff0133546ace P a152c25bb71949a0a62dce1e0476c6d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:50:21.179286 29392 raft_consensus.cc:729] T 8124b7f396634d2995d5ff0133546ace P a152c25bb71949a0a62dce1e0476c6d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a152c25bb71949a0a62dce1e0476c6d3, State: Initialized, Role: FOLLOWER
I20260806 22:50:21.179426 29392 consensus_queue.cc:260] T 8124b7f396634d2995d5ff0133546ace P a152c25bb71949a0a62dce1e0476c6d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a152c25bb71949a0a62dce1e0476c6d3" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 37655 } } peers { permanent_uuid: "baf570133f8749b68b951eaf0fcd9758" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 32857 } } peers { permanent_uuid: "3a75f07037a54584a1b8d1b3bd13336a" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 37891 } }
I20260806 22:50:21.179661 29392 ts_tablet_manager.cc:1428] T 8124b7f396634d2995d5ff0133546ace P a152c25bb71949a0a62dce1e0476c6d3: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:50:21.179733 29188 heartbeater.cc:503] Master 127.6.40.126:42633 was elected leader, sending a full tablet report...
W20260806 22:50:21.237485 29189 tablet.cc:2369] T 8124b7f396634d2995d5ff0133546ace P a152c25bb71949a0a62dce1e0476c6d3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:50:21.287854 29397 raft_consensus.cc:482] T 8124b7f396634d2995d5ff0133546ace P baf570133f8749b68b951eaf0fcd9758 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:50:21.287986 29397 raft_consensus.cc:504] T 8124b7f396634d2995d5ff0133546ace P baf570133f8749b68b951eaf0fcd9758 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a152c25bb71949a0a62dce1e0476c6d3" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 37655 } } peers { permanent_uuid: "baf570133f8749b68b951eaf0fcd9758" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 32857 } } peers { permanent_uuid: "3a75f07037a54584a1b8d1b3bd13336a" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 37891 } }
I20260806 22:50:21.288317 29397 leader_election.cc:288] T 8124b7f396634d2995d5ff0133546ace P baf570133f8749b68b951eaf0fcd9758 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a152c25bb71949a0a62dce1e0476c6d3 (127.6.40.66:37655), 3a75f07037a54584a1b8d1b3bd13336a (127.6.40.65:37891)
I20260806 22:50:21.291504 29380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:21.288550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.66:37655 (local address 127.6.40.67:39017)
0806 22:50:21.288632 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:21.288646 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:21.288701 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:21.288913 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:21.288914 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:21.288916 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:50:21.289022 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:21.289024 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:21.289735 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:21.289736 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:21.290631 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:21.290634 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:21.291239 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:21.291240 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:21.291338 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:21.291342 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:50:21.291405 (+ 63us) client_negotiation.cc:240] Negotiation successful
0806 22:50:21.291441 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260806 22:50:21.291744 29000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:21.288609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.67:58575 (local address 127.6.40.65:37891)
0806 22:50:21.288664 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:21.288665 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:21.288879 (+ 214us) server_negotiation.cc:407] Connection header received
0806 22:50:21.288944 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:21.288945 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:21.288961 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:21.289030 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:50:21.289226 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:21.289772 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:21.290779 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:21.291224 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:21.291307 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:21.291328 (+ 21us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:21.291577 (+ 249us) server_negotiation.cc:299] Negotiation successful
0806 22:50:21.291645 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 22:50:21.291608 29386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:21.288650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.67:39017 (local address 127.6.40.66:37655)
0806 22:50:21.288696 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:21.288698 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:21.288710 (+ 12us) server_negotiation.cc:407] Connection header received
0806 22:50:21.288790 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:21.288791 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:21.288805 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:21.288894 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:50:21.289116 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:21.289641 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:21.290740 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:21.291135 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:21.291195 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:21.291204 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:21.291458 (+ 254us) server_negotiation.cc:299] Negotiation successful
0806 22:50:21.291517 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 22:50:21.291837 29399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:21.288550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.65:37891 (local address 127.6.40.67:58575)
0806 22:50:21.288749 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:21.288760 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:21.288887 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:21.289041 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:21.289042 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:21.289043 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:50:21.289141 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:21.289143 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:21.289876 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:21.289877 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:21.290643 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:21.290645 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:21.291343 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:21.291345 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:21.291467 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:21.291472 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:50:21.291556 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 22:50:21.291734 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"mutex_wait_us":15,"thread_start_us":122,"threads_started":1}
I20260806 22:50:21.292176 29125 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8124b7f396634d2995d5ff0133546ace" candidate_uuid: "baf570133f8749b68b951eaf0fcd9758" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a152c25bb71949a0a62dce1e0476c6d3" is_pre_election: true
I20260806 22:50:21.292189 28918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8124b7f396634d2995d5ff0133546ace" candidate_uuid: "baf570133f8749b68b951eaf0fcd9758" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a75f07037a54584a1b8d1b3bd13336a" is_pre_election: true
I20260806 22:50:21.292430 28918 raft_consensus.cc:2454] T 8124b7f396634d2995d5ff0133546ace P 3a75f07037a54584a1b8d1b3bd13336a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate baf570133f8749b68b951eaf0fcd9758 in term 0.
I20260806 22:50:21.292438 29125 raft_consensus.cc:2454] T 8124b7f396634d2995d5ff0133546ace P a152c25bb71949a0a62dce1e0476c6d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate baf570133f8749b68b951eaf0fcd9758 in term 0.
I20260806 22:50:21.292727 29218 leader_election.cc:302] T 8124b7f396634d2995d5ff0133546ace P baf570133f8749b68b951eaf0fcd9758 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a152c25bb71949a0a62dce1e0476c6d3, baf570133f8749b68b951eaf0fcd9758; no voters:
I20260806 22:50:21.292918 29397 raft_consensus.cc:2781] T 8124b7f396634d2995d5ff0133546ace P baf570133f8749b68b951eaf0fcd9758 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:50:21.292989 29397 raft_consensus.cc:482] T 8124b7f396634d2995d5ff0133546ace P baf570133f8749b68b951eaf0fcd9758 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:50:21.293038 29397 raft_consensus.cc:3037] T 8124b7f396634d2995d5ff0133546ace P baf570133f8749b68b951eaf0fcd9758 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:50:21.293903 29397 raft_consensus.cc:504] T 8124b7f396634d2995d5ff0133546ace P baf570133f8749b68b951eaf0fcd9758 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a152c25bb71949a0a62dce1e0476c6d3" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 37655 } } peers { permanent_uuid: "baf570133f8749b68b951eaf0fcd9758" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 32857 } } peers { permanent_uuid: "3a75f07037a54584a1b8d1b3bd13336a" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 37891 } }
I20260806 22:50:21.294098 29397 leader_election.cc:288] T 8124b7f396634d2995d5ff0133546ace P baf570133f8749b68b951eaf0fcd9758 [CANDIDATE]: Term 1 election: Requested vote from peers a152c25bb71949a0a62dce1e0476c6d3 (127.6.40.66:37655), 3a75f07037a54584a1b8d1b3bd13336a (127.6.40.65:37891)
I20260806 22:50:21.294247 29125 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8124b7f396634d2995d5ff0133546ace" candidate_uuid: "baf570133f8749b68b951eaf0fcd9758" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a152c25bb71949a0a62dce1e0476c6d3"
I20260806 22:50:21.294291 28918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8124b7f396634d2995d5ff0133546ace" candidate_uuid: "baf570133f8749b68b951eaf0fcd9758" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a75f07037a54584a1b8d1b3bd13336a"
I20260806 22:50:21.294358 29125 raft_consensus.cc:3037] T 8124b7f396634d2995d5ff0133546ace P a152c25bb71949a0a62dce1e0476c6d3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:50:21.294389 28918 raft_consensus.cc:3037] T 8124b7f396634d2995d5ff0133546ace P 3a75f07037a54584a1b8d1b3bd13336a [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:50:21.295220 29125 raft_consensus.cc:2454] T 8124b7f396634d2995d5ff0133546ace P a152c25bb71949a0a62dce1e0476c6d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate baf570133f8749b68b951eaf0fcd9758 in term 1.
I20260806 22:50:21.295421 29218 leader_election.cc:302] T 8124b7f396634d2995d5ff0133546ace P baf570133f8749b68b951eaf0fcd9758 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a152c25bb71949a0a62dce1e0476c6d3, baf570133f8749b68b951eaf0fcd9758; no voters:
I20260806 22:50:21.295567 29397 raft_consensus.cc:2781] T 8124b7f396634d2995d5ff0133546ace P baf570133f8749b68b951eaf0fcd9758 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:50:21.295606 28918 raft_consensus.cc:2454] T 8124b7f396634d2995d5ff0133546ace P 3a75f07037a54584a1b8d1b3bd13336a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate baf570133f8749b68b951eaf0fcd9758 in term 1.
I20260806 22:50:21.295811 29397 raft_consensus.cc:686] T 8124b7f396634d2995d5ff0133546ace P baf570133f8749b68b951eaf0fcd9758 [term 1 LEADER]: Becoming Leader. State: Replica: baf570133f8749b68b951eaf0fcd9758, State: Running, Role: LEADER
I20260806 22:50:21.295931 29397 consensus_queue.cc:237] T 8124b7f396634d2995d5ff0133546ace P baf570133f8749b68b951eaf0fcd9758 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a152c25bb71949a0a62dce1e0476c6d3" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 37655 } } peers { permanent_uuid: "baf570133f8749b68b951eaf0fcd9758" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 32857 } } peers { permanent_uuid: "3a75f07037a54584a1b8d1b3bd13336a" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 37891 } }
I20260806 22:50:21.296967 28713 catalog_manager.cc:5495] T 8124b7f396634d2995d5ff0133546ace P baf570133f8749b68b951eaf0fcd9758 reported cstate change: term changed from 0 to 1, leader changed from <none> to baf570133f8749b68b951eaf0fcd9758 (127.6.40.67). New cstate: current_term: 1 leader_uuid: "baf570133f8749b68b951eaf0fcd9758" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a152c25bb71949a0a62dce1e0476c6d3" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 37655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "baf570133f8749b68b951eaf0fcd9758" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 32857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a75f07037a54584a1b8d1b3bd13336a" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 37891 } health_report { overall_health: UNKNOWN } } }
I20260806 22:50:21.315364 29389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:21.312456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52558 (local address 127.6.40.67:32857)
0806 22:50:21.312525 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:21.312527 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:21.312536 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:50:21.312589 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:21.312590 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:21.312612 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:21.312709 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:50:21.312930 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:21.313493 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:21.314371 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:21.314598 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:21.314691 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:21.314898 (+ 207us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:50:21.315032 (+ 134us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:50:21.315153 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:21.315180 (+ 27us) server_negotiation.cc:299] Negotiation successful
0806 22:50:21.315208 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 22:50:21.317965 29125 raft_consensus.cc:1261] T 8124b7f396634d2995d5ff0133546ace P a152c25bb71949a0a62dce1e0476c6d3 [term 1 FOLLOWER]: Refusing update from remote peer baf570133f8749b68b951eaf0fcd9758: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:50:21.317965 28918 raft_consensus.cc:1261] T 8124b7f396634d2995d5ff0133546ace P 3a75f07037a54584a1b8d1b3bd13336a [term 1 FOLLOWER]: Refusing update from remote peer baf570133f8749b68b951eaf0fcd9758: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:50:21.318398 29400 consensus_queue.cc:1035] T 8124b7f396634d2995d5ff0133546ace P baf570133f8749b68b951eaf0fcd9758 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a152c25bb71949a0a62dce1e0476c6d3" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 37655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:50:21.318553 29397 consensus_queue.cc:1035] T 8124b7f396634d2995d5ff0133546ace P baf570133f8749b68b951eaf0fcd9758 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a75f07037a54584a1b8d1b3bd13336a" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 37891 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:50:21.321251 29404 mvcc.cc:205] Tried to move back new op lower bound from 7315687920915087360 to 7315687920828915712. Current Snapshot: MvccSnapshot[applied={T|T < 7315687920915087360}]
I20260806 22:50:21.323158 28713 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52648:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:50:21.323261 28713 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52648:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:50:21.324030 28713 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a7f7cb97a9a4b75b6c39701b1d3e39f: Sending DeleteTablet for 3 replicas of tablet 8124b7f396634d2995d5ff0133546ace
I20260806 22:50:21.324342 28887 tablet_service.cc:1514] Processing DeleteTablet for tablet 8124b7f396634d2995d5ff0133546ace with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:50:21 UTC) from {username='kudu', principal='kudu/127.6.40.126@KRBTEST.COM'} at 127.0.0.1:53020
I20260806 22:50:21.324374 29094 tablet_service.cc:1514] Processing DeleteTablet for tablet 8124b7f396634d2995d5ff0133546ace with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:50:21 UTC) from {username='kudu', principal='kudu/127.6.40.126@KRBTEST.COM'} at 127.0.0.1:33622
I20260806 22:50:21.324465 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 28766
I20260806 22:50:21.324447 29283 tablet_service.cc:1514] Processing DeleteTablet for tablet 8124b7f396634d2995d5ff0133546ace with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:50:21 UTC) from {username='kudu', principal='kudu/127.6.40.126@KRBTEST.COM'} at 127.0.0.1:52552
I20260806 22:50:21.324587 29003 tablet_replica.cc:330] T 8124b7f396634d2995d5ff0133546ace P 3a75f07037a54584a1b8d1b3bd13336a: stopping tablet replica
I20260806 22:50:21.324646 29420 tablet_replica.cc:330] T 8124b7f396634d2995d5ff0133546ace P baf570133f8749b68b951eaf0fcd9758: stopping tablet replica
I20260806 22:50:21.324651 29003 raft_consensus.cc:2229] T 8124b7f396634d2995d5ff0133546ace P 3a75f07037a54584a1b8d1b3bd13336a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:50:21.324699 29003 raft_consensus.cc:2258] T 8124b7f396634d2995d5ff0133546ace P 3a75f07037a54584a1b8d1b3bd13336a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:50:21.324730 29420 raft_consensus.cc:2229] T 8124b7f396634d2995d5ff0133546ace P baf570133f8749b68b951eaf0fcd9758 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:50:21.324815 29420 raft_consensus.cc:2258] T 8124b7f396634d2995d5ff0133546ace P baf570133f8749b68b951eaf0fcd9758 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:50:21.328066 29420 ts_tablet_manager.cc:1905] T 8124b7f396634d2995d5ff0133546ace P baf570133f8749b68b951eaf0fcd9758: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:50:21.329882 29420 ts_tablet_manager.cc:1918] T 8124b7f396634d2995d5ff0133546ace P baf570133f8749b68b951eaf0fcd9758: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:50:21.329953 29420 log.cc:1196] T 8124b7f396634d2995d5ff0133546ace P baf570133f8749b68b951eaf0fcd9758: Deleting WAL directory at /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-2/wal/wals/8124b7f396634d2995d5ff0133546ace
I20260806 22:50:21.330264 29420 ts_tablet_manager.cc:1939] T 8124b7f396634d2995d5ff0133546ace P baf570133f8749b68b951eaf0fcd9758: Deleting consensus metadata
I20260806 22:50:21.330724 28696 catalog_manager.cc:4841] TS baf570133f8749b68b951eaf0fcd9758 (127.6.40.67:32857): tablet 8124b7f396634d2995d5ff0133546ace (table test-table [id=92621089f0604836a15b7e1fd2037111]) successfully deleted
I20260806 22:50:21.336970 29419 tablet_replica.cc:330] T 8124b7f396634d2995d5ff0133546ace P a152c25bb71949a0a62dce1e0476c6d3: stopping tablet replica
I20260806 22:50:21.337107 29419 raft_consensus.cc:2229] T 8124b7f396634d2995d5ff0133546ace P a152c25bb71949a0a62dce1e0476c6d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:50:21.337220 29419 raft_consensus.cc:2258] T 8124b7f396634d2995d5ff0133546ace P a152c25bb71949a0a62dce1e0476c6d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:50:21.337767 29419 ts_tablet_manager.cc:1905] T 8124b7f396634d2995d5ff0133546ace P a152c25bb71949a0a62dce1e0476c6d3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:50:21.339886 29419 ts_tablet_manager.cc:1918] T 8124b7f396634d2995d5ff0133546ace P a152c25bb71949a0a62dce1e0476c6d3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:50:21.339967 29419 log.cc:1196] T 8124b7f396634d2995d5ff0133546ace P a152c25bb71949a0a62dce1e0476c6d3: Deleting WAL directory at /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-1/wal/wals/8124b7f396634d2995d5ff0133546ace
I20260806 22:50:21.340265 29419 ts_tablet_manager.cc:1939] T 8124b7f396634d2995d5ff0133546ace P a152c25bb71949a0a62dce1e0476c6d3: Deleting consensus metadata
I20260806 22:50:21.340729 28696 catalog_manager.cc:4841] TS a152c25bb71949a0a62dce1e0476c6d3 (127.6.40.66:37655): tablet 8124b7f396634d2995d5ff0133546ace (table test-table [id=92621089f0604836a15b7e1fd2037111]) successfully deleted
W20260806 22:50:21.340871 28695 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.40.65:37891 (error 108)
I20260806 22:50:21.341204 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 28996
I20260806 22:50:21.341384 29385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:21.341234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.65:37891 (local address 127.0.0.1:53022)
0806 22:50:21.341302 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:21.341344 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.65:37891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260806 22:50:21.352607 28695 catalog_manager.cc:4584] TS 3a75f07037a54584a1b8d1b3bd13336a (127.6.40.65:37891): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8124b7f396634d2995d5ff0133546ace: Network error: Client connection negotiation failed: client connection to 127.6.40.65:37891: connect: Connection refused (error 111)
I20260806 22:50:21.361936 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 29195
I20260806 22:50:21.370113 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 28670
I20260806 22:50:41.381794 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.126:42633
--webserver_interface=127.6.40.126
--webserver_port=43717
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab
--principal=kudu/127.6.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:36473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.40.126:42633
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:50:41.465950 29422 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:50:41.466162 29422 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:50:41.466185 29422 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:50:41.467339 29422 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 22:50:41.467381 29422 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 22:50:41.467396 29422 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 22:50:41.467411 29422 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:50:41.467429 29422 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:50:41.467489 29422 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:50:41.467509 29422 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:50:41.467530 29422 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:50:41.469084 29422 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:36473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.6.40.126:42633
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab
--principal=kudu/127.6.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.126:42633
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.40.126
--webserver_port=43717
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29422
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:50:41.469347 29422 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:50:41.469604 29422 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 22:50:41.475075 29422 server_base.cc:1034] running on GCE node
W20260806 22:50:41.474988 29429 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:41.474999 29428 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:41.474989 29431 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:41.475597 29422 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:50:41.475978 29422 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:50:41.477128 29422 hybrid_clock.cc:648] HybridClock initialized: now 1786056641477117 us; error 30 us; skew 500 ppm
Aug 06 22:50:41 dist-test-slave-zh2d krb5kdc[27902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056641, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:50:41.479638 29422 init.cc:377] Logged in from keytab as kudu/127.6.40.126@KRBTEST.COM (short username kudu)
I20260806 22:50:41.480077 29422 webserver.cc:460] Webserver started at http://127.6.40.126:43717/ using document root <none> and password file <none>
I20260806 22:50:41.480324 29422 fs_manager.cc:359] Metadata directory not provided
I20260806 22:50:41.480396 29422 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:50:41.481838 29422 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:50:41.482513 29442 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:41.482686 29422 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:50:41.482796 29422 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/master-0/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/master-0/wal
uuid: "0a7f7cb97a9a4b75b6c39701b1d3e39f"
format_stamp: "Formatted at 2026-08-06 22:49:59 on dist-test-slave-zh2d"
server_key: "b909af0f5c21d28eae37cac377290833"
server_key_iv: "a755d12d30bf58a14981a6b7044aff99"
server_key_version: "encryptionkey@0"
I20260806 22:50:41.483142 29422 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:50:41.492461 29422 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:50:41.494554 29422 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:50:41.494727 29422 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:50:41.499003 29422 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.126:42633
I20260806 22:50:41.499053 29515 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.126:42633 every 8 connection(s)
I20260806 22:50:41.499439 29422 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
I20260806 22:50:41.500989 29518 sys_catalog.cc:263] Verifying existing consensus state
I20260806 22:50:41.501669 29518 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a7f7cb97a9a4b75b6c39701b1d3e39f: Bootstrap starting.
I20260806 22:50:41.508019 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 29422
I20260806 22:50:41.508412 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.65:37891
--local_ip_for_outbound_sockets=127.6.40.65
--tserver_master_addrs=127.6.40.126:42633
--webserver_port=34111
--webserver_interface=127.6.40.65
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.65.keytab
--principal=kudu/127.6.40.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:36473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:50:41.512128 29518 log.cc:824] T 00000000000000000000000000000000 P 0a7f7cb97a9a4b75b6c39701b1d3e39f: Log is configured to *not* fsync() on all Append() calls
I20260806 22:50:41.514794 29518 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a7f7cb97a9a4b75b6c39701b1d3e39f: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 22:50:41.515141 29518 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a7f7cb97a9a4b75b6c39701b1d3e39f: Bootstrap complete.
I20260806 22:50:41.517616 29518 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0a7f7cb97a9a4b75b6c39701b1d3e39f [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a7f7cb97a9a4b75b6c39701b1d3e39f" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 42633 } }
I20260806 22:50:41.517869 29518 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0a7f7cb97a9a4b75b6c39701b1d3e39f [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a7f7cb97a9a4b75b6c39701b1d3e39f, State: Initialized, Role: FOLLOWER
I20260806 22:50:41.518088 29518 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0a7f7cb97a9a4b75b6c39701b1d3e39f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a7f7cb97a9a4b75b6c39701b1d3e39f" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 42633 } }
I20260806 22:50:41.518191 29518 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0a7f7cb97a9a4b75b6c39701b1d3e39f [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:50:41.518247 29518 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0a7f7cb97a9a4b75b6c39701b1d3e39f [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:50:41.518296 29518 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0a7f7cb97a9a4b75b6c39701b1d3e39f [term 2 FOLLOWER]: Advancing to term 3
I20260806 22:50:41.519306 29518 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0a7f7cb97a9a4b75b6c39701b1d3e39f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a7f7cb97a9a4b75b6c39701b1d3e39f" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 42633 } }
I20260806 22:50:41.519430 29518 leader_election.cc:302] T 00000000000000000000000000000000 P 0a7f7cb97a9a4b75b6c39701b1d3e39f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0a7f7cb97a9a4b75b6c39701b1d3e39f; no voters:
I20260806 22:50:41.519654 29518 leader_election.cc:288] T 00000000000000000000000000000000 P 0a7f7cb97a9a4b75b6c39701b1d3e39f [CANDIDATE]: Term 3 election: Requested vote from peers
I20260806 22:50:41.519701 29523 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0a7f7cb97a9a4b75b6c39701b1d3e39f [term 3 FOLLOWER]: Leader election won for term 3
I20260806 22:50:41.519843 29523 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0a7f7cb97a9a4b75b6c39701b1d3e39f [term 3 LEADER]: Becoming Leader. State: Replica: 0a7f7cb97a9a4b75b6c39701b1d3e39f, State: Running, Role: LEADER
I20260806 22:50:41.519944 29518 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0a7f7cb97a9a4b75b6c39701b1d3e39f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:50:41.519932 29523 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0a7f7cb97a9a4b75b6c39701b1d3e39f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a7f7cb97a9a4b75b6c39701b1d3e39f" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 42633 } }
I20260806 22:50:41.520287 29523 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a7f7cb97a9a4b75b6c39701b1d3e39f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0a7f7cb97a9a4b75b6c39701b1d3e39f. Latest consensus state: current_term: 3 leader_uuid: "0a7f7cb97a9a4b75b6c39701b1d3e39f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a7f7cb97a9a4b75b6c39701b1d3e39f" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 42633 } } }
I20260806 22:50:41.520364 29523 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a7f7cb97a9a4b75b6c39701b1d3e39f [sys.catalog]: This master's current role is: LEADER
I20260806 22:50:41.520524 29525 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a7f7cb97a9a4b75b6c39701b1d3e39f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "0a7f7cb97a9a4b75b6c39701b1d3e39f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a7f7cb97a9a4b75b6c39701b1d3e39f" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 42633 } } }
I20260806 22:50:41.520601 29533 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:50:41.521433 29525 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a7f7cb97a9a4b75b6c39701b1d3e39f [sys.catalog]: This master's current role is: LEADER
I20260806 22:50:41.521525 29533 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:50:41.521632 29533 catalog_manager.cc:1259] Loaded cluster ID: b6dca6ddc3014539b3d05e506340117b
I20260806 22:50:41.521665 29533 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:50:41.523048 29533 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:50:41.523229 29533 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 0a7f7cb97a9a4b75b6c39701b1d3e39f: Loaded TSK: 0
I20260806 22:50:41.523272 29533 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 0a7f7cb97a9a4b75b6c39701b1d3e39f: Loaded TSK: 1
F20260806 22:50:41.523518 29533 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fee249e69ad google::LogMessage::Fail() at ??:0
@ 0x7fee249e988a google::LogMessage::SendToLog() at ??:0
@ 0x7fee249e64de google::LogMessage::Flush() at ??:0
@ 0x7fee249e7a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fee27153104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fee27150d41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7fee2c9441cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7fee2c94448e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7fee2c944594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7fee2c90ac48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7fee2c914ad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fee268b40d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fee268a9954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fee2770d6db start_thread at ??:0
@ 0x7fee234e461f clone at ??:0
W20260806 22:50:41.617514 29520 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:50:41.617722 29520 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:50:41.617746 29520 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:50:41.618901 29520 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:50:41.618942 29520 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:50:41.618969 29520 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.65
I20260806 22:50:41.620455 29520 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:36473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.65.keytab
--principal=kudu/127.6.40.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.65:37891
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.40.65
--webserver_port=34111
--tserver_master_addrs=127.6.40.126:42633
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29520
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.65
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:50:41.620698 29520 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:50:41.620985 29520 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:50:41.627239 29579 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:41.627261 29578 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:41.627504 29581 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:41.627591 29520 server_base.cc:1034] running on GCE node
I20260806 22:50:41.627839 29520 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:50:41.628129 29520 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:50:41.629321 29520 hybrid_clock.cc:648] HybridClock initialized: now 1786056641629314 us; error 36 us; skew 500 ppm
Aug 06 22:50:41 dist-test-slave-zh2d krb5kdc[27902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056641, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:50:41.632464 29520 init.cc:377] Logged in from keytab as kudu/127.6.40.65@KRBTEST.COM (short username kudu)
I20260806 22:50:41.633040 29520 webserver.cc:460] Webserver started at http://127.6.40.65:34111/ using document root <none> and password file <none>
I20260806 22:50:41.633288 29520 fs_manager.cc:359] Metadata directory not provided
I20260806 22:50:41.633360 29520 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:50:41.634819 29520 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:50:41.635573 29594 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:41.635748 29520 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:50:41.635851 29520 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-0/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-0/wal
uuid: "3a75f07037a54584a1b8d1b3bd13336a"
format_stamp: "Formatted at 2026-08-06 22:50:00 on dist-test-slave-zh2d"
server_key: "25aba12f593f79340ee874c9a3eb7765"
server_key_iv: "e23616f1a04dbb5dbf1f790b1201bbde"
server_key_version: "encryptionkey@0"
I20260806 22:50:41.636214 29520 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:50:41.657584 29520 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:50:41.659886 29520 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:50:41.660068 29520 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:50:41.660331 29520 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:50:41.660807 29604 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 22:50:41.662935 29520 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 22:50:41.663002 29520 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260806 22:50:41.663046 29520 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 22:50:41.663726 29520 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 22:50:41.663784 29520 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260806 22:50:41.663858 29604 tablet_bootstrap.cc:492] T 8124b7f396634d2995d5ff0133546ace P 3a75f07037a54584a1b8d1b3bd13336a: Bootstrap starting.
I20260806 22:50:41.671279 29520 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.65:37891
I20260806 22:50:41.671783 29520 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
I20260806 22:50:41.674633 29756 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.65:37891 every 8 connection(s)
I20260806 22:50:41.675498 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 29520
I20260806 22:50:41.675916 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.66:37655
--local_ip_for_outbound_sockets=127.6.40.66
--tserver_master_addrs=127.6.40.126:42633
--webserver_port=33437
--webserver_interface=127.6.40.66
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.66.keytab
--principal=kudu/127.6.40.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:36473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:50:41.680408 29604 log.cc:824] T 8124b7f396634d2995d5ff0133546ace P 3a75f07037a54584a1b8d1b3bd13336a: Log is configured to *not* fsync() on all Append() calls
I20260806 22:50:41.683830 29604 tablet_bootstrap.cc:492] T 8124b7f396634d2995d5ff0133546ace P 3a75f07037a54584a1b8d1b3bd13336a: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 22:50:41.684222 29604 tablet_bootstrap.cc:492] T 8124b7f396634d2995d5ff0133546ace P 3a75f07037a54584a1b8d1b3bd13336a: Bootstrap complete.
I20260806 22:50:41.685807 29604 ts_tablet_manager.cc:1397] T 8124b7f396634d2995d5ff0133546ace P 3a75f07037a54584a1b8d1b3bd13336a: Time spent bootstrapping tablet: real 0.022s user 0.004s sys 0.013s
I20260806 22:50:41.687709 29604 raft_consensus.cc:348] T 8124b7f396634d2995d5ff0133546ace P 3a75f07037a54584a1b8d1b3bd13336a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a152c25bb71949a0a62dce1e0476c6d3" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 37655 } } peers { permanent_uuid: "baf570133f8749b68b951eaf0fcd9758" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 32857 } } peers { permanent_uuid: "3a75f07037a54584a1b8d1b3bd13336a" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 37891 } }
I20260806 22:50:41.688134 29604 raft_consensus.cc:729] T 8124b7f396634d2995d5ff0133546ace P 3a75f07037a54584a1b8d1b3bd13336a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a75f07037a54584a1b8d1b3bd13336a, State: Initialized, Role: FOLLOWER
I20260806 22:50:41.688319 29604 consensus_queue.cc:260] T 8124b7f396634d2995d5ff0133546ace P 3a75f07037a54584a1b8d1b3bd13336a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a152c25bb71949a0a62dce1e0476c6d3" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 37655 } } peers { permanent_uuid: "baf570133f8749b68b951eaf0fcd9758" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 32857 } } peers { permanent_uuid: "3a75f07037a54584a1b8d1b3bd13336a" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 37891 } }
I20260806 22:50:41.688645 29604 ts_tablet_manager.cc:1428] T 8124b7f396634d2995d5ff0133546ace P 3a75f07037a54584a1b8d1b3bd13336a: Time spent starting tablet: real 0.003s user 0.001s sys 0.003s
W20260806 22:50:41.737247 29760 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:50:41.675320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:42633 (local address 127.6.40.65:48631)
0806 22:50:41.676689 (+ 1369us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:41.676776 (+ 87us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:41.678126 (+ 1350us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:41.736990 (+ 58864us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:42633: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":1258,"thread_start_us":116,"threads_started":1}
W20260806 22:50:41.737576 29758 heartbeater.cc:644] Failed to heartbeat to 127.6.40.126:42633 (0 consecutive failures): Network error: Failed to ping master at 127.6.40.126:42633: Client connection negotiation failed: client connection to 127.6.40.126:42633: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 22:50:41.737962 29760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:41.737831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:42633 (local address 127.6.40.65:33913)
0806 22:50:41.737887 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:41.737938 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:42633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260806 22:50:41.738400 29760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:41.738272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:42633 (local address 127.6.40.65:53375)
0806 22:50:41.738348 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:41.738380 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:42633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260806 22:50:41.738574 29758 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 22:50:41.766846 29761 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:50:41.767053 29761 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:50:41.767076 29761 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:50:41.768229 29761 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:50:41.768271 29761 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:50:41.768325 29761 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.66
I20260806 22:50:41.769860 29761 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:36473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.66.keytab
--principal=kudu/127.6.40.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.66:37655
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.40.66
--webserver_port=33437
--tserver_master_addrs=127.6.40.126:42633
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29761
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.66
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:50:41.770102 29761 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:50:41.770337 29761 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:50:41.775483 29770 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:41.775463 29771 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:41.775454 29773 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:41.775727 29761 server_base.cc:1034] running on GCE node
I20260806 22:50:41.776002 29761 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:50:41.776269 29761 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:50:41.777437 29761 hybrid_clock.cc:648] HybridClock initialized: now 1786056641777416 us; error 38 us; skew 500 ppm
Aug 06 22:50:41 dist-test-slave-zh2d krb5kdc[27902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056641, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:50:41.779897 29761 init.cc:377] Logged in from keytab as kudu/127.6.40.66@KRBTEST.COM (short username kudu)
I20260806 22:50:41.780313 29761 webserver.cc:460] Webserver started at http://127.6.40.66:33437/ using document root <none> and password file <none>
I20260806 22:50:41.780543 29761 fs_manager.cc:359] Metadata directory not provided
I20260806 22:50:41.780614 29761 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:50:41.782001 29761 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:50:41.782603 29782 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:41.782820 29761 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:50:41.782905 29761 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-1/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-1/wal
uuid: "a152c25bb71949a0a62dce1e0476c6d3"
format_stamp: "Formatted at 2026-08-06 22:50:00 on dist-test-slave-zh2d"
server_key: "9b0dcf3af3447da2fadf70f640bed9a7"
server_key_iv: "bd8b2fbc787a5c9153913649d60b0be1"
server_key_version: "encryptionkey@0"
I20260806 22:50:41.783197 29761 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:50:41.796238 29761 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:50:41.798538 29761 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:50:41.798642 29761 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:50:41.798825 29761 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:50:41.799139 29761 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:50:41.799194 29761 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:41.799237 29761 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:50:41.799265 29761 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:41.805213 29761 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.66:37655
I20260806 22:50:41.805292 29934 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.66:37655 every 8 connection(s)
I20260806 22:50:41.805640 29761 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
I20260806 22:50:41.806707 29937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:41.806326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:42633 (local address 127.6.40.66:34633)
0806 22:50:41.806488 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:41.806564 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:42633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":79,"threads_started":1}
W20260806 22:50:41.807124 29935 heartbeater.cc:644] Failed to heartbeat to 127.6.40.126:42633 (0 consecutive failures): Network error: Failed to ping master at 127.6.40.126:42633: Client connection negotiation failed: client connection to 127.6.40.126:42633: connect: Connection refused (error 111)
I20260806 22:50:41.807416 29937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:41.807332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:42633 (local address 127.6.40.66:53893)
0806 22:50:41.807374 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:41.807399 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:42633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260806 22:50:41.807708 29937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:41.807633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:42633 (local address 127.6.40.66:58029)
0806 22:50:41.807672 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:41.807693 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:42633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260806 22:50:41.807837 29935 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 22:50:41.813225 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 29761
I20260806 22:50:41.813475 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.67:32857
--local_ip_for_outbound_sockets=127.6.40.67
--tserver_master_addrs=127.6.40.126:42633
--webserver_port=46549
--webserver_interface=127.6.40.67
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.67.keytab
--principal=kudu/127.6.40.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:36473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:50:41.898705 29940 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:50:41.898931 29940 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:50:41.898957 29940 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:50:41.900110 29940 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:50:41.900146 29940 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:50:41.900173 29940 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.67
I20260806 22:50:41.901638 29940 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:36473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.67.keytab
--principal=kudu/127.6.40.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.67:32857
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.40.67
--webserver_port=46549
--tserver_master_addrs=127.6.40.126:42633
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29940
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.67
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:50:41.901836 29940 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:50:41.902046 29940 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:50:41.907466 29948 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:41.907488 29951 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:41.907553 29949 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:41.907871 29940 server_base.cc:1034] running on GCE node
I20260806 22:50:41.908062 29940 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:50:41.908310 29940 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:50:41.909458 29940 hybrid_clock.cc:648] HybridClock initialized: now 1786056641909430 us; error 45 us; skew 500 ppm
Aug 06 22:50:41 dist-test-slave-zh2d krb5kdc[27902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056641, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:50:41.912091 29940 init.cc:377] Logged in from keytab as kudu/127.6.40.67@KRBTEST.COM (short username kudu)
I20260806 22:50:41.912563 29940 webserver.cc:460] Webserver started at http://127.6.40.67:46549/ using document root <none> and password file <none>
I20260806 22:50:41.912773 29940 fs_manager.cc:359] Metadata directory not provided
I20260806 22:50:41.912814 29940 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:50:41.914227 29940 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:50:41.914815 29962 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:41.915030 29940 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:50:41.915123 29940 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-2/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-2/wal
uuid: "baf570133f8749b68b951eaf0fcd9758"
format_stamp: "Formatted at 2026-08-06 22:50:00 on dist-test-slave-zh2d"
server_key: "65b9da73df9edc56555e857cec9777c2"
server_key_iv: "91a59b4ea6bcf5c50a1dd0b7036f0620"
server_key_version: "encryptionkey@0"
I20260806 22:50:41.915465 29940 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:50:41.929492 29940 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:50:41.931360 29940 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:50:41.931526 29940 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:50:41.931771 29940 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:50:41.932140 29940 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:50:41.932175 29940 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:41.932197 29940 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:50:41.932243 29940 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:41.938392 29940 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.67:32857
I20260806 22:50:41.938444 30110 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.67:32857 every 8 connection(s)
I20260806 22:50:41.938812 29940 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
I20260806 22:50:41.939364 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 29940
I20260806 22:50:41.940091 30113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:41.939570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:42633 (local address 127.6.40.67:60391)
0806 22:50:41.939766 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:41.939874 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:42633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":89,"threads_started":1}
W20260806 22:50:41.940515 30111 heartbeater.cc:644] Failed to heartbeat to 127.6.40.126:42633 (0 consecutive failures): Network error: Failed to ping master at 127.6.40.126:42633: Client connection negotiation failed: client connection to 127.6.40.126:42633: connect: Connection refused (error 111)
I20260806 22:50:41.940775 30113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:41.940698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:42633 (local address 127.6.40.67:40673)
0806 22:50:41.940729 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:41.940756 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:42633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
I20260806 22:50:41.941080 30113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:41.940989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:42633 (local address 127.6.40.67:49465)
0806 22:50:41.941040 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:41.941063 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:42633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260806 22:50:41.941195 30111 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 22:50:42.739286 30117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:42.738998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:42633 (local address 127.6.40.65:40159)
0806 22:50:42.739164 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:42.739216 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:42633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":82,"threads_started":1}
I20260806 22:50:42.808339 30118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:42.808135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:42633 (local address 127.6.40.66:38217)
0806 22:50:42.808252 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:42.808286 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:42633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":64,"threads_started":1}
I20260806 22:50:42.941706 30119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:42.941492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:42633 (local address 127.6.40.67:43243)
0806 22:50:42.941615 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:42.941649 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:42633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":63,"threads_started":1}
I20260806 22:50:43.285524 30120 raft_consensus.cc:482] T 8124b7f396634d2995d5ff0133546ace P 3a75f07037a54584a1b8d1b3bd13336a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:50:43.285763 30120 raft_consensus.cc:504] T 8124b7f396634d2995d5ff0133546ace P 3a75f07037a54584a1b8d1b3bd13336a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a152c25bb71949a0a62dce1e0476c6d3" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 37655 } } peers { permanent_uuid: "baf570133f8749b68b951eaf0fcd9758" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 32857 } } peers { permanent_uuid: "3a75f07037a54584a1b8d1b3bd13336a" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 37891 } }
I20260806 22:50:43.286170 30120 leader_election.cc:288] T 8124b7f396634d2995d5ff0133546ace P 3a75f07037a54584a1b8d1b3bd13336a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a152c25bb71949a0a62dce1e0476c6d3 (127.6.40.66:37655), baf570133f8749b68b951eaf0fcd9758 (127.6.40.67:32857)
Aug 06 22:50:43 dist-test-slave-zh2d krb5kdc[27902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056641, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.65@KRBTEST.COM for kudu/127.6.40.67@KRBTEST.COM
Aug 06 22:50:43 dist-test-slave-zh2d krb5kdc[27902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056641, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.65@KRBTEST.COM for kudu/127.6.40.66@KRBTEST.COM
I20260806 22:50:43.303462 30121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:43.286424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.66:37655 (local address 127.6.40.65:45647)
0806 22:50:43.286581 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:43.286592 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:43.286690 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:43.287651 (+ 961us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:43.287654 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:43.288157 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:43.288414 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:43.288421 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:43.289632 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:43.289633 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:43.290399 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:43.290402 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:43.291029 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:43.291030 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:43.291136 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:43.297785 (+ 6649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:43.297795 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:43.299916 (+ 2121us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:50:43.301842 (+ 1926us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:43.301843 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:43.301843 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:43.302104 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:43.302367 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:43.302368 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:43.302368 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:43.302489 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:43.302895 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:50:43.302903 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:50:43.303038 (+ 135us) client_negotiation.cc:769] Sending connection context
0806 22:50:43.303148 (+ 110us) client_negotiation.cc:240] Negotiation successful
0806 22:50:43.303352 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"mutex_wait_us":53,"thread_start_us":93,"threads_started":1}
I20260806 22:50:43.303476 30124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:43.286826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.65:48613 (local address 127.6.40.67:32857)
0806 22:50:43.286976 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:43.286988 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:43.287040 (+ 52us) server_negotiation.cc:407] Connection header received
0806 22:50:43.287473 (+ 433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:43.287475 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:43.287539 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:43.287688 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:43.288565 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:43.289479 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:43.290516 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:43.290916 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:43.291034 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:43.299656 (+ 8622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:43.299661 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:43.299662 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:43.299677 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:43.301621 (+ 1944us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:43.302201 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:43.302205 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:43.302205 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:43.302300 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:43.302566 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:43.302567 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:43.302567 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:43.302823 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:43.302922 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:43.303220 (+ 298us) server_negotiation.cc:299] Negotiation successful
0806 22:50:43.303356 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":87,"threads_started":1}
I20260806 22:50:43.303763 30123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:43.286649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.67:32857 (local address 127.6.40.65:48613)
0806 22:50:43.286760 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:43.286839 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:43.286902 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:43.287703 (+ 801us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:43.287704 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:43.288157 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:43.288415 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:43.288421 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:43.289588 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:43.289590 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:43.290400 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:43.290403 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:43.291016 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:43.291017 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:43.291135 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:43.297800 (+ 6665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:43.297805 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:43.299518 (+ 1713us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:50:43.301712 (+ 2194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:43.301720 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:43.301723 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:43.302105 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:43.302391 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:43.302392 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:43.302392 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:43.302492 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:43.302921 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:50:43.302922 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:50:43.303037 (+ 115us) client_negotiation.cc:769] Sending connection context
0806 22:50:43.303148 (+ 111us) client_negotiation.cc:240] Negotiation successful
0806 22:50:43.303447 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"mutex_wait_us":141,"thread_start_us":50,"threads_started":1}
I20260806 22:50:43.303495 30122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:43.286585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.65:45647 (local address 127.6.40.66:37655)
0806 22:50:43.286785 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:43.286797 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:43.286848 (+ 51us) server_negotiation.cc:407] Connection header received
0806 22:50:43.287390 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:43.287392 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:43.287439 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:43.287565 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:43.288554 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:43.289497 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:43.290492 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:43.290919 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:43.291054 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:43.300039 (+ 8985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:43.300048 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:43.300048 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:43.300060 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:43.301752 (+ 1692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:43.302196 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:43.302200 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:43.302201 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:43.302296 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:43.302566 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:43.302567 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:43.302567 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:43.302820 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:43.302910 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:43.303220 (+ 310us) server_negotiation.cc:299] Negotiation successful
0806 22:50:43.303373 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":67,"threads_started":1}
W20260806 22:50:43.304677 29600 leader_election.cc:341] T 8124b7f396634d2995d5ff0133546ace P 3a75f07037a54584a1b8d1b3bd13336a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer baf570133f8749b68b951eaf0fcd9758 (127.6.40.67:32857): Not found: Tablet not found: 8124b7f396634d2995d5ff0133546ace
W20260806 22:50:43.304915 29600 leader_election.cc:341] T 8124b7f396634d2995d5ff0133546ace P 3a75f07037a54584a1b8d1b3bd13336a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a152c25bb71949a0a62dce1e0476c6d3 (127.6.40.66:37655): Not found: Tablet not found: 8124b7f396634d2995d5ff0133546ace
I20260806 22:50:43.304998 29600 leader_election.cc:302] T 8124b7f396634d2995d5ff0133546ace P 3a75f07037a54584a1b8d1b3bd13336a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a75f07037a54584a1b8d1b3bd13336a; no voters: a152c25bb71949a0a62dce1e0476c6d3, baf570133f8749b68b951eaf0fcd9758
I20260806 22:50:43.305127 30120 raft_consensus.cc:2726] T 8124b7f396634d2995d5ff0133546ace P 3a75f07037a54584a1b8d1b3bd13336a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 22:50:43.739925 30123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:43.739773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:42633 (local address 127.6.40.65:41401)
0806 22:50:43.739842 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:43.739882 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:42633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260806 22:50:43.809073 30125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:43.808803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:42633 (local address 127.6.40.66:42173)
0806 22:50:43.809003 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:43.809038 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:42633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":112,"threads_started":1}
I20260806 22:50:43.942389 30126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:43.942183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:42633 (local address 127.6.40.67:44169)
0806 22:50:43.942312 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:43.942347 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:42633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":74,"threads_started":1}
I20260806 22:50:44.740638 30127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:44.740350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:42633 (local address 127.6.40.65:43435)
0806 22:50:44.740524 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:44.740566 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:42633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":89,"threads_started":1}
I20260806 22:50:44.809849 30129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:44.809584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:42633 (local address 127.6.40.66:60219)
0806 22:50:44.809749 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:44.809798 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:42633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":70,"threads_started":1}
I20260806 22:50:44.858907 30130 raft_consensus.cc:482] T 8124b7f396634d2995d5ff0133546ace P 3a75f07037a54584a1b8d1b3bd13336a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:50:44.859007 30130 raft_consensus.cc:504] T 8124b7f396634d2995d5ff0133546ace P 3a75f07037a54584a1b8d1b3bd13336a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a152c25bb71949a0a62dce1e0476c6d3" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 37655 } } peers { permanent_uuid: "baf570133f8749b68b951eaf0fcd9758" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 32857 } } peers { permanent_uuid: "3a75f07037a54584a1b8d1b3bd13336a" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 37891 } }
I20260806 22:50:44.859136 30130 leader_election.cc:288] T 8124b7f396634d2995d5ff0133546ace P 3a75f07037a54584a1b8d1b3bd13336a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a152c25bb71949a0a62dce1e0476c6d3 (127.6.40.66:37655), baf570133f8749b68b951eaf0fcd9758 (127.6.40.67:32857)
W20260806 22:50:44.859637 29600 leader_election.cc:341] T 8124b7f396634d2995d5ff0133546ace P 3a75f07037a54584a1b8d1b3bd13336a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a152c25bb71949a0a62dce1e0476c6d3 (127.6.40.66:37655): Not found: Tablet not found: 8124b7f396634d2995d5ff0133546ace
W20260806 22:50:44.859726 29600 leader_election.cc:341] T 8124b7f396634d2995d5ff0133546ace P 3a75f07037a54584a1b8d1b3bd13336a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer baf570133f8749b68b951eaf0fcd9758 (127.6.40.67:32857): Not found: Tablet not found: 8124b7f396634d2995d5ff0133546ace
I20260806 22:50:44.859772 29600 leader_election.cc:302] T 8124b7f396634d2995d5ff0133546ace P 3a75f07037a54584a1b8d1b3bd13336a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a75f07037a54584a1b8d1b3bd13336a; no voters: a152c25bb71949a0a62dce1e0476c6d3, baf570133f8749b68b951eaf0fcd9758
I20260806 22:50:44.859867 30130 raft_consensus.cc:2726] T 8124b7f396634d2995d5ff0133546ace P 3a75f07037a54584a1b8d1b3bd13336a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 22:50:44.943143 30131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:44.942919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:42633 (local address 127.6.40.67:37765)
0806 22:50:44.943060 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:44.943100 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:42633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":69,"threads_started":1}
I20260806 22:50:45.741400 30133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:45.741158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:42633 (local address 127.6.40.65:45693)
0806 22:50:45.741307 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:45.741347 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:42633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":68,"threads_started":1}
I20260806 22:50:45.810531 30136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:45.810321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:42633 (local address 127.6.40.66:59829)
0806 22:50:45.810448 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:45.810486 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:42633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":62,"threads_started":1}
I20260806 22:50:45.943892 30137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:45.943671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:42633 (local address 127.6.40.67:43199)
0806 22:50:45.943807 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:45.943848 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:42633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":62,"threads_started":1}
I20260806 22:50:46.635913 30138 raft_consensus.cc:482] T 8124b7f396634d2995d5ff0133546ace P 3a75f07037a54584a1b8d1b3bd13336a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:50:46.636050 30138 raft_consensus.cc:504] T 8124b7f396634d2995d5ff0133546ace P 3a75f07037a54584a1b8d1b3bd13336a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a152c25bb71949a0a62dce1e0476c6d3" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 37655 } } peers { permanent_uuid: "baf570133f8749b68b951eaf0fcd9758" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 32857 } } peers { permanent_uuid: "3a75f07037a54584a1b8d1b3bd13336a" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 37891 } }
I20260806 22:50:46.636379 30138 leader_election.cc:288] T 8124b7f396634d2995d5ff0133546ace P 3a75f07037a54584a1b8d1b3bd13336a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a152c25bb71949a0a62dce1e0476c6d3 (127.6.40.66:37655), baf570133f8749b68b951eaf0fcd9758 (127.6.40.67:32857)
W20260806 22:50:46.636869 29600 leader_election.cc:341] T 8124b7f396634d2995d5ff0133546ace P 3a75f07037a54584a1b8d1b3bd13336a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a152c25bb71949a0a62dce1e0476c6d3 (127.6.40.66:37655): Not found: Tablet not found: 8124b7f396634d2995d5ff0133546ace
W20260806 22:50:46.636947 29600 leader_election.cc:341] T 8124b7f396634d2995d5ff0133546ace P 3a75f07037a54584a1b8d1b3bd13336a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer baf570133f8749b68b951eaf0fcd9758 (127.6.40.67:32857): Not found: Tablet not found: 8124b7f396634d2995d5ff0133546ace
I20260806 22:50:46.636965 29600 leader_election.cc:302] T 8124b7f396634d2995d5ff0133546ace P 3a75f07037a54584a1b8d1b3bd13336a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a75f07037a54584a1b8d1b3bd13336a; no voters: a152c25bb71949a0a62dce1e0476c6d3, baf570133f8749b68b951eaf0fcd9758
I20260806 22:50:46.637053 30138 raft_consensus.cc:2726] T 8124b7f396634d2995d5ff0133546ace P 3a75f07037a54584a1b8d1b3bd13336a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 22:50:46.742236 30140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:46.741963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:42633 (local address 127.6.40.65:46653)
0806 22:50:46.742142 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:46.742186 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:42633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":82,"threads_started":1}
I20260806 22:50:46.811234 30141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:46.811016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:42633 (local address 127.6.40.66:41931)
0806 22:50:46.811158 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:46.811196 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:42633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":65,"threads_started":1}
I20260806 22:50:46.944765 30142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:46.944482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:42633 (local address 127.6.40.67:53181)
0806 22:50:46.944678 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:46.944722 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:42633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":98,"threads_started":1}
I20260806 22:50:47.743001 30144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:47.742759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:42633 (local address 127.6.40.65:38837)
0806 22:50:47.742907 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:47.742948 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:42633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":77,"threads_started":1}
I20260806 22:50:47.812178 30145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:47.811886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:42633 (local address 127.6.40.66:59877)
0806 22:50:47.812062 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:47.812117 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:42633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":117,"threads_started":1}
I20260806 22:50:47.945602 30146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:47.945304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:42633 (local address 127.6.40.67:46559)
0806 22:50:47.945494 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:47.945543 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:42633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":128,"threads_started":1}
I20260806 22:50:48.743884 30148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:48.743607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:42633 (local address 127.6.40.65:34043)
0806 22:50:48.743774 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:48.743828 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:42633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":90,"threads_started":1}
I20260806 22:50:48.812928 30149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:48.812623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:42633 (local address 127.6.40.66:54199)
0806 22:50:48.812785 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:48.812820 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:42633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":104,"threads_started":1}
I20260806 22:50:48.946271 30152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:48.946052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:42633 (local address 127.6.40.67:57841)
0806 22:50:48.946188 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:48.946226 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:42633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":69,"threads_started":1}
I20260806 22:50:49.744671 30153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:49.744414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:42633 (local address 127.6.40.65:44865)
0806 22:50:49.744549 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:49.744594 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:42633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":83,"threads_started":1}
I20260806 22:50:49.813791 30155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:49.813532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:42633 (local address 127.6.40.66:55713)
0806 22:50:49.813692 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:49.813735 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:42633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":65,"threads_started":1}
I20260806 22:50:49.947013 30156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:49.946783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:42633 (local address 127.6.40.67:47013)
0806 22:50:49.946923 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:49.946965 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:42633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":64,"threads_started":1}
I20260806 22:50:50.745375 30157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:50.745149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:42633 (local address 127.6.40.65:39249)
0806 22:50:50.745285 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:50.745324 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:42633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":70,"threads_started":1}
I20260806 22:50:50.814538 30158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:50.814276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:42633 (local address 127.6.40.66:47425)
0806 22:50:50.814441 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:50.814478 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:42633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":63,"threads_started":1}
I20260806 22:50:50.947726 30160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:50.947489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:42633 (local address 127.6.40.67:55491)
0806 22:50:50.947633 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:50.947676 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:42633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":71,"threads_started":1}
I20260806 22:50:51.483615 30161 raft_consensus.cc:482] T 8124b7f396634d2995d5ff0133546ace P 3a75f07037a54584a1b8d1b3bd13336a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:50:51.483727 30161 raft_consensus.cc:504] T 8124b7f396634d2995d5ff0133546ace P 3a75f07037a54584a1b8d1b3bd13336a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a152c25bb71949a0a62dce1e0476c6d3" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 37655 } } peers { permanent_uuid: "baf570133f8749b68b951eaf0fcd9758" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 32857 } } peers { permanent_uuid: "3a75f07037a54584a1b8d1b3bd13336a" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 37891 } }
I20260806 22:50:51.483973 30161 leader_election.cc:288] T 8124b7f396634d2995d5ff0133546ace P 3a75f07037a54584a1b8d1b3bd13336a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a152c25bb71949a0a62dce1e0476c6d3 (127.6.40.66:37655), baf570133f8749b68b951eaf0fcd9758 (127.6.40.67:32857)
W20260806 22:50:51.484453 29600 leader_election.cc:341] T 8124b7f396634d2995d5ff0133546ace P 3a75f07037a54584a1b8d1b3bd13336a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a152c25bb71949a0a62dce1e0476c6d3 (127.6.40.66:37655): Not found: Tablet not found: 8124b7f396634d2995d5ff0133546ace
W20260806 22:50:51.484547 29600 leader_election.cc:341] T 8124b7f396634d2995d5ff0133546ace P 3a75f07037a54584a1b8d1b3bd13336a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer baf570133f8749b68b951eaf0fcd9758 (127.6.40.67:32857): Not found: Tablet not found: 8124b7f396634d2995d5ff0133546ace
I20260806 22:50:51.484586 29600 leader_election.cc:302] T 8124b7f396634d2995d5ff0133546ace P 3a75f07037a54584a1b8d1b3bd13336a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a75f07037a54584a1b8d1b3bd13336a; no voters: a152c25bb71949a0a62dce1e0476c6d3, baf570133f8749b68b951eaf0fcd9758
I20260806 22:50:51.484688 30161 raft_consensus.cc:2726] T 8124b7f396634d2995d5ff0133546ace P 3a75f07037a54584a1b8d1b3bd13336a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 22:50:51.746136 30162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:51.745903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:42633 (local address 127.6.40.65:51891)
0806 22:50:51.746046 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:51.746086 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:42633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":81,"threads_started":1}
I20260806 22:50:51.815279 30164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:51.815061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:42633 (local address 127.6.40.66:56051)
0806 22:50:51.815191 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:51.815227 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:42633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":67,"threads_started":1}
I20260806 22:50:51.948561 30166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:51.948305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:42633 (local address 127.6.40.67:35451)
0806 22:50:51.948457 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:51.948497 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:42633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":67,"threads_started":1}
I20260806 22:50:52.746924 30167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:52.746669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:42633 (local address 127.6.40.65:38487)
0806 22:50:52.746830 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:52.746872 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:42633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":76,"threads_started":1}
I20260806 22:50:52.816064 30168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:52.815852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:42633 (local address 127.6.40.66:44175)
0806 22:50:52.815985 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:52.816021 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:42633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":71,"threads_started":1}
I20260806 22:50:52.949405 30169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:52.949145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:42633 (local address 127.6.40.67:33859)
0806 22:50:52.949316 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:52.949357 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:42633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":88,"threads_started":1}
I20260806 22:50:53.747655 30170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:53.747399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:42633 (local address 127.6.40.65:38639)
0806 22:50:53.747572 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:53.747605 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:42633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":87,"threads_started":1}
I20260806 22:50:53.816938 30172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:53.816624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:42633 (local address 127.6.40.66:55511)
0806 22:50:53.816787 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:53.816826 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:42633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":75,"threads_started":1}
I20260806 22:50:53.950125 30174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:53.949892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:42633 (local address 127.6.40.67:43791)
0806 22:50:53.950037 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:53.950075 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:42633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":67,"threads_started":1}
I20260806 22:50:54.748404 30175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:54.748176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:42633 (local address 127.6.40.65:48483)
0806 22:50:54.748317 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:54.748348 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:42633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":77,"threads_started":1}
I20260806 22:50:54.817659 30176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:54.817436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:42633 (local address 127.6.40.66:34605)
0806 22:50:54.817568 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:54.817611 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:42633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":70,"threads_started":1}
I20260806 22:50:54.951013 30177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:54.950707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:42633 (local address 127.6.40.67:43747)
0806 22:50:54.950919 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:54.950962 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:42633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":85,"threads_started":1}
I20260806 22:50:55.749274 30180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:55.749001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:42633 (local address 127.6.40.65:48505)
0806 22:50:55.749178 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:55.749219 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:42633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":90,"threads_started":1}
I20260806 22:50:55.818348 30182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:55.818120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:42633 (local address 127.6.40.66:42977)
0806 22:50:55.818262 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:55.818302 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:42633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":78,"threads_started":1}
I20260806 22:50:55.951794 30183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:55.951543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:42633 (local address 127.6.40.67:41373)
0806 22:50:55.951701 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:55.951744 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:42633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":78,"threads_started":1}
I20260806 22:50:56.750105 30184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:56.749875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:42633 (local address 127.6.40.65:37369)
0806 22:50:56.750008 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:56.750050 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:42633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":70,"threads_started":1}
I20260806 22:50:56.819146 30185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:56.818915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:42633 (local address 127.6.40.66:36269)
0806 22:50:56.819065 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:56.819100 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:42633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":64,"threads_started":1}
I20260806 22:50:56.948077 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 29520
I20260806 22:50:56.956148 30186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:56.955888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:42633 (local address 127.6.40.67:35613)
0806 22:50:56.956046 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:56.956095 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.126:42633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":72,"threads_started":1}
I20260806 22:50:56.960934 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 29761
I20260806 22:50:56.972525 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 29940
2026-08-06T22:50:56Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58642 ms)
[----------] 30 tests from SecurityITest (251260 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:50:57 dist-test-slave-zh2d krb5kdc[30194](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:50:57 dist-test-slave-zh2d krb5kdc[30194](info): set up 2 sockets
Aug 06 22:50:57 dist-test-slave-zh2d krb5kdc[30194](info): commencing operation
krb5kdc: starting...
W20260806 22:50:58.198580 6305 mini_kdc.cc:121] Time spent starting KDC: real 1.200s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/krb5kdc/test-user.keytab.
Aug 06 22:50:58 dist-test-slave-zh2d krb5kdc[30194](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056658, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:50:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:50:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.40.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.40.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.40.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab.
Entry for principal kudu/127.6.40.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab.
I20260806 22:50:58.340938 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.126:39671
--webserver_interface=127.6.40.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab
--principal=kudu/127.6.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:42113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.40.126:39671
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:50:58.425565 30207 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:50:58.425814 30207 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:50:58.425851 30207 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:50:58.427011 30207 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:50:58.427081 30207 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:50:58.427107 30207 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:50:58.427137 30207 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:50:58.427166 30207 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:50:58.428651 30207 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:42113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.40.126:39671
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab
--principal=kudu/127.6.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.126:39671
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30207
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:50:58.428942 30207 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:50:58.429173 30207 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:50:58.434639 30218 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:58.434718 30207 server_base.cc:1034] running on GCE node
W20260806 22:50:58.434696 30216 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:58.434715 30214 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:58.435165 30207 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:50:58.435503 30207 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:50:58.436681 30207 hybrid_clock.cc:648] HybridClock initialized: now 1786056658436658 us; error 35 us; skew 500 ppm
Aug 06 22:50:58 dist-test-slave-zh2d krb5kdc[30194](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056658, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:50:58.439249 30207 init.cc:377] Logged in from keytab as kudu/127.6.40.126@KRBTEST.COM (short username kudu)
I20260806 22:50:58.439651 30207 webserver.cc:460] Webserver started at http://127.6.40.126:46719/ using document root <none> and password file <none>
I20260806 22:50:58.439834 30207 fs_manager.cc:359] Metadata directory not provided
I20260806 22:50:58.439873 30207 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:50:58.439961 30207 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:50:58.441054 30207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/minicluster-data/master-0/data/instance:
uuid: "21cdf9be9e95482cb228b878ef4e48f6"
format_stamp: "Formatted at 2026-08-06 22:50:58 on dist-test-slave-zh2d"
server_key: "89a5821ceb707db39dfb114cd2183551"
server_key_iv: "af13e4e63877fb4b458f43490ff240cc"
server_key_version: "encryptionkey@0"
I20260806 22:50:58.441365 30207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/minicluster-data/master-0/wal/instance:
uuid: "21cdf9be9e95482cb228b878ef4e48f6"
format_stamp: "Formatted at 2026-08-06 22:50:58 on dist-test-slave-zh2d"
server_key: "89a5821ceb707db39dfb114cd2183551"
server_key_iv: "af13e4e63877fb4b458f43490ff240cc"
server_key_version: "encryptionkey@0"
I20260806 22:50:58.442662 30207 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:50:58.443359 30225 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:58.443574 30207 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:50:58.443662 30207 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/minicluster-data/master-0/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/minicluster-data/master-0/wal
uuid: "21cdf9be9e95482cb228b878ef4e48f6"
format_stamp: "Formatted at 2026-08-06 22:50:58 on dist-test-slave-zh2d"
server_key: "89a5821ceb707db39dfb114cd2183551"
server_key_iv: "af13e4e63877fb4b458f43490ff240cc"
server_key_version: "encryptionkey@0"
I20260806 22:50:58.443717 30207 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:50:58.475699 30207 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:50:58.478072 30207 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:50:58.478190 30207 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:50:58.482404 30207 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.126:39671
I20260806 22:50:58.482488 30303 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.126:39671 every 8 connection(s)
I20260806 22:50:58.482831 30207 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
I20260806 22:50:58.483804 30305 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:50:58.486239 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 30207
I20260806 22:50:58.486246 30305 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21cdf9be9e95482cb228b878ef4e48f6: Bootstrap starting.
I20260806 22:50:58.486331 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/minicluster-data/master-0/wal/instance
I20260806 22:50:58.486564 6305 external_mini_cluster.cc:1428] Setting key a38fa836c15a5799b7d13b66f8321f7b
I20260806 22:50:58.486990 30305 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 21cdf9be9e95482cb228b878ef4e48f6: Neither blocks nor log segments found. Creating new log.
I20260806 22:50:58.487421 30305 log.cc:824] T 00000000000000000000000000000000 P 21cdf9be9e95482cb228b878ef4e48f6: Log is configured to *not* fsync() on all Append() calls
I20260806 22:50:58.488245 30305 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21cdf9be9e95482cb228b878ef4e48f6: No bootstrap required, opened a new log
I20260806 22:50:58.490650 30305 raft_consensus.cc:348] T 00000000000000000000000000000000 P 21cdf9be9e95482cb228b878ef4e48f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21cdf9be9e95482cb228b878ef4e48f6" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 39671 } }
I20260806 22:50:58.490815 30305 raft_consensus.cc:374] T 00000000000000000000000000000000 P 21cdf9be9e95482cb228b878ef4e48f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:50:58.490842 30305 raft_consensus.cc:729] T 00000000000000000000000000000000 P 21cdf9be9e95482cb228b878ef4e48f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21cdf9be9e95482cb228b878ef4e48f6, State: Initialized, Role: FOLLOWER
I20260806 22:50:58.490975 30305 consensus_queue.cc:260] T 00000000000000000000000000000000 P 21cdf9be9e95482cb228b878ef4e48f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21cdf9be9e95482cb228b878ef4e48f6" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 39671 } }
I20260806 22:50:58.491060 30305 raft_consensus.cc:388] T 00000000000000000000000000000000 P 21cdf9be9e95482cb228b878ef4e48f6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:50:58.491107 30305 raft_consensus.cc:482] T 00000000000000000000000000000000 P 21cdf9be9e95482cb228b878ef4e48f6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:50:58.491155 30305 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 21cdf9be9e95482cb228b878ef4e48f6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:50:58.491815 30305 raft_consensus.cc:504] T 00000000000000000000000000000000 P 21cdf9be9e95482cb228b878ef4e48f6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21cdf9be9e95482cb228b878ef4e48f6" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 39671 } }
I20260806 22:50:58.491927 30305 leader_election.cc:302] T 00000000000000000000000000000000 P 21cdf9be9e95482cb228b878ef4e48f6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 21cdf9be9e95482cb228b878ef4e48f6; no voters:
I20260806 22:50:58.492089 30305 leader_election.cc:288] T 00000000000000000000000000000000 P 21cdf9be9e95482cb228b878ef4e48f6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:50:58.492159 30311 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 21cdf9be9e95482cb228b878ef4e48f6 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:50:58.492362 30305 sys_catalog.cc:564] T 00000000000000000000000000000000 P 21cdf9be9e95482cb228b878ef4e48f6 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:50:58.492347 30311 raft_consensus.cc:686] T 00000000000000000000000000000000 P 21cdf9be9e95482cb228b878ef4e48f6 [term 1 LEADER]: Becoming Leader. State: Replica: 21cdf9be9e95482cb228b878ef4e48f6, State: Running, Role: LEADER
I20260806 22:50:58.492630 30311 consensus_queue.cc:237] T 00000000000000000000000000000000 P 21cdf9be9e95482cb228b878ef4e48f6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21cdf9be9e95482cb228b878ef4e48f6" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 39671 } }
I20260806 22:50:58.493072 30314 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21cdf9be9e95482cb228b878ef4e48f6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 21cdf9be9e95482cb228b878ef4e48f6. Latest consensus state: current_term: 1 leader_uuid: "21cdf9be9e95482cb228b878ef4e48f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21cdf9be9e95482cb228b878ef4e48f6" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 39671 } } }
I20260806 22:50:58.493160 30314 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21cdf9be9e95482cb228b878ef4e48f6 [sys.catalog]: This master's current role is: LEADER
I20260806 22:50:58.493371 30323 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:50:58.493412 30313 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21cdf9be9e95482cb228b878ef4e48f6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "21cdf9be9e95482cb228b878ef4e48f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21cdf9be9e95482cb228b878ef4e48f6" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 39671 } } }
I20260806 22:50:58.493562 30313 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21cdf9be9e95482cb228b878ef4e48f6 [sys.catalog]: This master's current role is: LEADER
I20260806 22:50:58.493969 30323 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 06 22:50:58 dist-test-slave-zh2d krb5kdc[30194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056658, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
I20260806 22:50:58.495633 30323 catalog_manager.cc:1347] Generated new cluster ID: 9da379bf69944696a56629eea4831367
I20260806 22:50:58.495690 30323 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:50:58.499514 30308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:58.487166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53304 (local address 127.6.40.126:39671)
0806 22:50:58.487383 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:58.487395 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:58.487429 (+ 34us) server_negotiation.cc:407] Connection header received
0806 22:50:58.488043 (+ 614us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:58.488053 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:58.488120 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:58.488276 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:58.489091 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:58.490576 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:58.491575 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:58.492230 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:58.492696 (+ 466us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:58.495680 (+ 2984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:58.495684 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:58.495685 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:58.495698 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:58.497787 (+ 2089us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:58.498204 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:58.498208 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:58.498208 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:58.498324 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:58.498542 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:58.498542 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:58.498543 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:58.498876 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:58.499048 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:58.499142 (+ 94us) server_negotiation.cc:299] Negotiation successful
0806 22:50:58.499293 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":91,"threads_started":1}
I20260806 22:50:58.506394 30323 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:50:58.506933 30323 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:50:58.510687 30323 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 21cdf9be9e95482cb228b878ef4e48f6: Generated new TSK 0
I20260806 22:50:58.510923 30323 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.40.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.40.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.40.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.65.keytab.
Entry for principal kudu/127.6.40.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.65.keytab.
I20260806 22:50:58.541769 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.65:0
--local_ip_for_outbound_sockets=127.6.40.65
--webserver_interface=127.6.40.65
--webserver_port=0
--tserver_master_addrs=127.6.40.126:39671
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.65.keytab
--principal=kudu/127.6.40.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:42113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:50:58.643318 30337 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:50:58.643527 30337 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:50:58.643549 30337 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:50:58.644675 30337 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:50:58.644714 30337 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:50:58.644740 30337 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.65
I20260806 22:50:58.646241 30337 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:42113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.65.keytab
--principal=kudu/127.6.40.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.40.65
--webserver_port=0
--tserver_master_addrs=127.6.40.126:39671
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30337
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.65
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:50:58.646636 30337 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:50:58.646870 30337 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:50:58.652745 30346 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:58.652753 30343 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:58.653136 30337 server_base.cc:1034] running on GCE node
W20260806 22:50:58.652906 30342 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:58.653373 30337 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:50:58.653607 30337 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:50:58.654770 30337 hybrid_clock.cc:648] HybridClock initialized: now 1786056658654755 us; error 31 us; skew 500 ppm
Aug 06 22:50:58 dist-test-slave-zh2d krb5kdc[30194](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056658, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:50:58.657387 30337 init.cc:377] Logged in from keytab as kudu/127.6.40.65@KRBTEST.COM (short username kudu)
I20260806 22:50:58.657789 30337 webserver.cc:460] Webserver started at http://127.6.40.65:37657/ using document root <none> and password file <none>
I20260806 22:50:58.657971 30337 fs_manager.cc:359] Metadata directory not provided
I20260806 22:50:58.658035 30337 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:50:58.658133 30337 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:50:58.659152 30337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/minicluster-data/ts-0/data/instance:
uuid: "4c36bf9d914a4476a33be2dab1bd03f1"
format_stamp: "Formatted at 2026-08-06 22:50:58 on dist-test-slave-zh2d"
server_key: "90233e81bf5ec86524224ea55ee77fff"
server_key_iv: "d3b4ea4179149e930d40107c582e854d"
server_key_version: "encryptionkey@0"
I20260806 22:50:58.659456 30337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/minicluster-data/ts-0/wal/instance:
uuid: "4c36bf9d914a4476a33be2dab1bd03f1"
format_stamp: "Formatted at 2026-08-06 22:50:58 on dist-test-slave-zh2d"
server_key: "90233e81bf5ec86524224ea55ee77fff"
server_key_iv: "d3b4ea4179149e930d40107c582e854d"
server_key_version: "encryptionkey@0"
I20260806 22:50:58.660809 30337 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 22:50:58.661628 30359 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:58.661796 30337 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:50:58.661896 30337 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/minicluster-data/ts-0/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/minicluster-data/ts-0/wal
uuid: "4c36bf9d914a4476a33be2dab1bd03f1"
format_stamp: "Formatted at 2026-08-06 22:50:58 on dist-test-slave-zh2d"
server_key: "90233e81bf5ec86524224ea55ee77fff"
server_key_iv: "d3b4ea4179149e930d40107c582e854d"
server_key_version: "encryptionkey@0"
I20260806 22:50:58.661976 30337 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:50:58.678596 30337 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:50:58.680601 30337 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:50:58.680775 30337 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:50:58.681087 30337 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:50:58.681411 30337 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:50:58.681463 30337 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:58.681500 30337 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:50:58.681529 30337 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:58.687634 30337 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.65:38865
I20260806 22:50:58.687708 30514 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.65:38865 every 8 connection(s)
I20260806 22:50:58.688093 30337 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
Aug 06 22:50:58 dist-test-slave-zh2d krb5kdc[30194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056658, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.65@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
I20260806 22:50:58.697351 30308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:58.688912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.65:56671 (local address 127.6.40.126:39671)
0806 22:50:58.688980 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:58.688983 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:58.689153 (+ 170us) server_negotiation.cc:407] Connection header received
0806 22:50:58.689636 (+ 483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:58.689637 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:58.689648 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:58.689697 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:58.690632 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:58.691151 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:58.692148 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:58.692390 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:58.692442 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:58.694725 (+ 2283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:58.694727 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:58.694727 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:58.694734 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:58.696152 (+ 1418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:58.696525 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:58.696525 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:58.696526 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:58.696577 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:58.696783 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:58.696784 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:58.696784 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:58.696896 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:58.696946 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:58.697200 (+ 254us) server_negotiation.cc:299] Negotiation successful
0806 22:50:58.697276 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 22:50:58.697510 30518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:58.688856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:39671 (local address 127.6.40.65:56671)
0806 22:50:58.689026 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:58.689064 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:58.689554 (+ 490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:58.689756 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:58.689760 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:58.690340 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:58.690551 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:58.690556 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:58.691244 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:58.691245 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:58.692069 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:58.692071 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:58.692455 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:58.692456 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:58.692579 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:58.693119 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:58.693129 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:58.694640 (+ 1511us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:50:58.696228 (+ 1588us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:58.696235 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:58.696237 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:58.696457 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:58.696634 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:58.696635 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:58.696635 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:58.696733 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:58.696940 (+ 207us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:50:58.696945 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:50:58.697070 (+ 125us) client_negotiation.cc:769] Sending connection context
0806 22:50:58.697157 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 22:50:58.697284 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":95,"threads_started":1}
I20260806 22:50:58.698065 30516 heartbeater.cc:347] Connected to a master server at 127.6.40.126:39671
I20260806 22:50:58.698180 30516 heartbeater.cc:464] Registering TS with master...
I20260806 22:50:58.698230 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 30337
I20260806 22:50:58.698343 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/minicluster-data/ts-0/wal/instance
I20260806 22:50:58.698428 30516 heartbeater.cc:511] Master 127.6.40.126:39671 requested a full tablet report, sending...
I20260806 22:50:58.698594 6305 external_mini_cluster.cc:1428] Setting key ba0914ab9574e24f0e08648f74cd55d5
I20260806 22:50:58.699056 30246 ts_manager.cc:194] Registered new tserver with Master: 4c36bf9d914a4476a33be2dab1bd03f1 (127.6.40.65:38865)
I20260806 22:50:58.699669 30246 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.40.65@KRBTEST.COM'} at 127.6.40.65:56671
WARNING: no policy specified for kudu/127.6.40.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.40.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.40.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.66.keytab.
Entry for principal kudu/127.6.40.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.66.keytab.
I20260806 22:50:58.723613 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.66:0
--local_ip_for_outbound_sockets=127.6.40.66
--webserver_interface=127.6.40.66
--webserver_port=0
--tserver_master_addrs=127.6.40.126:39671
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.66.keytab
--principal=kudu/127.6.40.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:42113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:50:58.807015 30521 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:50:58.807255 30521 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:50:58.807293 30521 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:50:58.808437 30521 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:50:58.808501 30521 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:50:58.808545 30521 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.66
I20260806 22:50:58.809990 30521 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:42113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.66.keytab
--principal=kudu/127.6.40.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.40.66
--webserver_port=0
--tserver_master_addrs=127.6.40.126:39671
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30521
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.66
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:50:58.810251 30521 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:50:58.810475 30521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:50:58.815747 30527 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:58.815861 30521 server_base.cc:1034] running on GCE node
W20260806 22:50:58.815791 30526 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:58.815748 30529 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:58.816183 30521 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:50:58.816399 30521 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:50:58.817543 30521 hybrid_clock.cc:648] HybridClock initialized: now 1786056658817522 us; error 39 us; skew 500 ppm
Aug 06 22:50:58 dist-test-slave-zh2d krb5kdc[30194](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056658, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:50:58.820111 30521 init.cc:377] Logged in from keytab as kudu/127.6.40.66@KRBTEST.COM (short username kudu)
I20260806 22:50:58.820566 30521 webserver.cc:460] Webserver started at http://127.6.40.66:37929/ using document root <none> and password file <none>
I20260806 22:50:58.820782 30521 fs_manager.cc:359] Metadata directory not provided
I20260806 22:50:58.820827 30521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:50:58.821017 30521 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:50:58.822044 30521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/minicluster-data/ts-1/data/instance:
uuid: "95cf1a2e3de74bc590be69c531c3e077"
format_stamp: "Formatted at 2026-08-06 22:50:58 on dist-test-slave-zh2d"
server_key: "b44a19e8cb621a98aef16603e7fef279"
server_key_iv: "05ccfe757136b57a15d24bace1f37816"
server_key_version: "encryptionkey@0"
I20260806 22:50:58.822407 30521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/minicluster-data/ts-1/wal/instance:
uuid: "95cf1a2e3de74bc590be69c531c3e077"
format_stamp: "Formatted at 2026-08-06 22:50:58 on dist-test-slave-zh2d"
server_key: "b44a19e8cb621a98aef16603e7fef279"
server_key_iv: "05ccfe757136b57a15d24bace1f37816"
server_key_version: "encryptionkey@0"
I20260806 22:50:58.823716 30521 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:50:58.824424 30542 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:58.824590 30521 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:50:58.824702 30521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/minicluster-data/ts-1/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/minicluster-data/ts-1/wal
uuid: "95cf1a2e3de74bc590be69c531c3e077"
format_stamp: "Formatted at 2026-08-06 22:50:58 on dist-test-slave-zh2d"
server_key: "b44a19e8cb621a98aef16603e7fef279"
server_key_iv: "05ccfe757136b57a15d24bace1f37816"
server_key_version: "encryptionkey@0"
I20260806 22:50:58.824770 30521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:50:58.841548 30521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:50:58.843643 30521 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:50:58.843822 30521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:50:58.844087 30521 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:50:58.844414 30521 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:50:58.844467 30521 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:58.844521 30521 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:50:58.844549 30521 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:58.851055 30521 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.66:42075
I20260806 22:50:58.851104 30698 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.66:42075 every 8 connection(s)
I20260806 22:50:58.851413 30521 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
Aug 06 22:50:58 dist-test-slave-zh2d krb5kdc[30194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056658, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.66@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
I20260806 22:50:58.858829 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 30521
I20260806 22:50:58.858970 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/minicluster-data/ts-1/wal/instance
I20260806 22:50:58.859172 6305 external_mini_cluster.cc:1428] Setting key 9e6033c2e14830b284db4c29cdd4d853
I20260806 22:50:58.860908 30308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:58.852080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.66:46437 (local address 127.6.40.126:39671)
0806 22:50:58.852124 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:58.852126 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:58.852354 (+ 228us) server_negotiation.cc:407] Connection header received
0806 22:50:58.852939 (+ 585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:58.852940 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:58.852954 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:58.853009 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:58.853978 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:58.854502 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:58.855512 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:58.855713 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:58.855797 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:58.857931 (+ 2134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:58.857933 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:58.857934 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:58.857942 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:58.859347 (+ 1405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:58.859828 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:58.859830 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:58.859830 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:58.859898 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:58.860165 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:58.860166 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:58.860166 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:58.860316 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:58.860390 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:58.860638 (+ 248us) server_negotiation.cc:299] Negotiation successful
0806 22:50:58.860742 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 22:50:58.860946 30701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:58.852062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:39671 (local address 127.6.40.66:46437)
0806 22:50:58.852231 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:58.852268 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:58.852795 (+ 527us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:58.853097 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:58.853101 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:58.853674 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:58.853872 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:58.853877 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:58.854602 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:58.854603 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:58.855416 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:58.855418 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:58.855811 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:58.855812 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:58.855941 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:58.856412 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:58.856421 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:58.857835 (+ 1414us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:50:58.859423 (+ 1588us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:58.859431 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:58.859434 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:58.859736 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:58.859975 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:58.859976 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:58.859977 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:58.860097 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:58.860384 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:50:58.860388 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:50:58.860508 (+ 120us) client_negotiation.cc:769] Sending connection context
0806 22:50:58.860590 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 22:50:58.860720 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":81,"threads_started":1}
I20260806 22:50:58.862216 30699 heartbeater.cc:347] Connected to a master server at 127.6.40.126:39671
I20260806 22:50:58.862301 30699 heartbeater.cc:464] Registering TS with master...
I20260806 22:50:58.862499 30699 heartbeater.cc:511] Master 127.6.40.126:39671 requested a full tablet report, sending...
I20260806 22:50:58.862885 30246 ts_manager.cc:194] Registered new tserver with Master: 95cf1a2e3de74bc590be69c531c3e077 (127.6.40.66:42075)
I20260806 22:50:58.863408 30246 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.40.66@KRBTEST.COM'} at 127.6.40.66:46437
WARNING: no policy specified for kudu/127.6.40.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.40.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.40.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.67.keytab.
Entry for principal kudu/127.6.40.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.67.keytab.
I20260806 22:50:58.887557 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.67:0
--local_ip_for_outbound_sockets=127.6.40.67
--webserver_interface=127.6.40.67
--webserver_port=0
--tserver_master_addrs=127.6.40.126:39671
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.67.keytab
--principal=kudu/127.6.40.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:42113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:50:58.972527 30706 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:50:58.972741 30706 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:50:58.972765 30706 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:50:58.973965 30706 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:50:58.974012 30706 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:50:58.974040 30706 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.67
I20260806 22:50:58.975701 30706 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:42113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.67.keytab
--principal=kudu/127.6.40.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.40.67
--webserver_port=0
--tserver_master_addrs=127.6.40.126:39671
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30706
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.67
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:50:58.975946 30706 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:50:58.976184 30706 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:50:58.981593 30713 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:58.981600 30716 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:58.981630 30712 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:58.981663 30706 server_base.cc:1034] running on GCE node
I20260806 22:50:58.982028 30706 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:50:58.982291 30706 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:50:58.983454 30706 hybrid_clock.cc:648] HybridClock initialized: now 1786056658983438 us; error 31 us; skew 500 ppm
Aug 06 22:50:58 dist-test-slave-zh2d krb5kdc[30194](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056658, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:50:58.986022 30706 init.cc:377] Logged in from keytab as kudu/127.6.40.67@KRBTEST.COM (short username kudu)
I20260806 22:50:58.986502 30706 webserver.cc:460] Webserver started at http://127.6.40.67:34419/ using document root <none> and password file <none>
I20260806 22:50:58.986726 30706 fs_manager.cc:359] Metadata directory not provided
I20260806 22:50:58.986789 30706 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:50:58.986943 30706 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:50:58.988339 30706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/minicluster-data/ts-2/data/instance:
uuid: "97fac4545d5346d58e255673729c5aa0"
format_stamp: "Formatted at 2026-08-06 22:50:58 on dist-test-slave-zh2d"
server_key: "4a7e81118d4a854f5530d9053cad0a47"
server_key_iv: "5cc321f0abf53412c0cd68531ac28f64"
server_key_version: "encryptionkey@0"
I20260806 22:50:58.988709 30706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/minicluster-data/ts-2/wal/instance:
uuid: "97fac4545d5346d58e255673729c5aa0"
format_stamp: "Formatted at 2026-08-06 22:50:58 on dist-test-slave-zh2d"
server_key: "4a7e81118d4a854f5530d9053cad0a47"
server_key_iv: "5cc321f0abf53412c0cd68531ac28f64"
server_key_version: "encryptionkey@0"
I20260806 22:50:58.990079 30706 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:50:58.990837 30723 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:58.991012 30706 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:50:58.991112 30706 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/minicluster-data/ts-2/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/minicluster-data/ts-2/wal
uuid: "97fac4545d5346d58e255673729c5aa0"
format_stamp: "Formatted at 2026-08-06 22:50:58 on dist-test-slave-zh2d"
server_key: "4a7e81118d4a854f5530d9053cad0a47"
server_key_iv: "5cc321f0abf53412c0cd68531ac28f64"
server_key_version: "encryptionkey@0"
I20260806 22:50:58.991195 30706 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:50:59.009871 30706 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:50:59.011970 30706 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:50:59.012135 30706 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:50:59.012373 30706 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:50:59.012642 30706 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:50:59.012672 30706 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:59.012694 30706 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:50:59.012706 30706 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:59.018903 30706 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.67:42709
I20260806 22:50:59.018949 30882 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.67:42709 every 8 connection(s)
I20260806 22:50:59.019270 30706 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
I20260806 22:50:59.023048 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 30706
I20260806 22:50:59.023125 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/minicluster-data/ts-2/wal/instance
I20260806 22:50:59.023305 6305 external_mini_cluster.cc:1428] Setting key 6054ab3ba760af657f1af32f1687206d
Aug 06 22:50:59 dist-test-slave-zh2d krb5kdc[30194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056658, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.67@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
I20260806 22:50:59.030409 30308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:59.020033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.67:52273 (local address 127.6.40.126:39671)
0806 22:50:59.020129 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:59.020131 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:59.020319 (+ 188us) server_negotiation.cc:407] Connection header received
0806 22:50:59.021014 (+ 695us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:59.021016 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:59.021029 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:59.021092 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:59.022141 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:59.022755 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:59.024433 (+ 1678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:59.024662 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:59.024775 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:59.027196 (+ 2421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:59.027198 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:59.027198 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:59.027208 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:59.028964 (+ 1756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:59.029386 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:59.029387 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:59.029388 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:59.029477 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:59.029734 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:59.029735 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:59.029736 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:59.029851 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:59.029925 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:59.030210 (+ 285us) server_negotiation.cc:299] Negotiation successful
0806 22:50:59.030291 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 22:50:59.030588 30886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:59.019923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:39671 (local address 127.6.40.67:52273)
0806 22:50:59.020169 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:59.020216 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:59.020898 (+ 682us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:59.021170 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:59.021174 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:59.021819 (+ 645us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:59.022034 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:59.022040 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:59.022954 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:59.022956 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:59.024267 (+ 1311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:59.024271 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:59.024799 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:59.024801 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:59.024991 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:59.025589 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:59.025599 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:59.027083 (+ 1484us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:50:59.029065 (+ 1982us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:59.029074 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:59.029076 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:59.029307 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:59.029549 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:59.029550 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:59.029550 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:59.029662 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:59.029918 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:50:59.029927 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:50:59.030063 (+ 136us) client_negotiation.cc:769] Sending connection context
0806 22:50:59.030162 (+ 99us) client_negotiation.cc:240] Negotiation successful
0806 22:50:59.030346 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":118,"threads_started":1}
I20260806 22:50:59.031068 30883 heartbeater.cc:347] Connected to a master server at 127.6.40.126:39671
I20260806 22:50:59.031175 30883 heartbeater.cc:464] Registering TS with master...
I20260806 22:50:59.031352 30883 heartbeater.cc:511] Master 127.6.40.126:39671 requested a full tablet report, sending...
I20260806 22:50:59.031744 30246 ts_manager.cc:194] Registered new tserver with Master: 97fac4545d5346d58e255673729c5aa0 (127.6.40.67:42709)
I20260806 22:50:59.032112 30246 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.40.67@KRBTEST.COM'} at 127.6.40.67:52273
I20260806 22:50:59.035158 6305 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:50:59.041952 30308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:59.036524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53314 (local address 127.6.40.126:39671)
0806 22:50:59.036564 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:59.036566 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:59.036670 (+ 104us) server_negotiation.cc:407] Connection header received
0806 22:50:59.036714 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:59.036714 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:59.036722 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:59.036768 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:59.037338 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:59.037801 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:59.038485 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:59.038658 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:59.038707 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:59.039822 (+ 1115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:59.039823 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:59.039823 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:59.039828 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:59.041041 (+ 1213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:59.041339 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:59.041340 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:59.041340 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:59.041387 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:59.041541 (+ 154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:59.041541 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:59.041542 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:59.041672 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:59.041751 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:59.041834 (+ 83us) server_negotiation.cc:299] Negotiation successful
0806 22:50:59.041890 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 22:50:59.043546 30246 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53314:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:50:59.045205 30246 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:50:59.051759 30897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:59.047464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.67:42709 (local address 127.0.0.1:36088)
0806 22:50:59.047608 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:59.047654 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:59.047750 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:59.048433 (+ 683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:59.048436 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:59.048438 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:50:59.048508 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:59.048510 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:59.049514 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:59.049515 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:59.050666 (+ 1151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:59.050669 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:59.051449 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:59.051450 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:59.051542 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:59.051556 (+ 14us) client_negotiation.cc:769] Sending connection context
0806 22:50:59.051633 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 22:50:59.051683 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":80,"threads_started":1}
I20260806 22:50:59.051901 30900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:59.047597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36088 (local address 127.6.40.67:42709)
0806 22:50:59.048243 (+ 646us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:59.048248 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:59.048258 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:50:59.048298 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:59.048300 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:59.048316 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:59.048592 (+ 276us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:50:59.048668 (+ 76us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:59.049410 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:59.050745 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:59.051362 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:59.051439 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:59.051495 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:59.051659 (+ 164us) server_negotiation.cc:299] Negotiation successful
0806 22:50:59.051703 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":610,"thread_start_us":104,"threads_started":1}
I20260806 22:50:59.052093 30903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:59.047721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.65:38865 (local address 127.0.0.1:56946)
0806 22:50:59.048032 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:59.048044 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:59.048092 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:59.048538 (+ 446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:59.048539 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:59.048541 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:50:59.048579 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:59.048580 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:59.049959 (+ 1379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:59.049960 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:59.050942 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:59.050944 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:59.051848 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:59.051849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:59.051925 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:59.051930 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:50:59.051963 (+ 33us) client_negotiation.cc:240] Negotiation successful
0806 22:50:59.052014 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":85,"threads_started":1}
I20260806 22:50:59.052415 30902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:59.047868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56946 (local address 127.6.40.65:38865)
0806 22:50:59.048343 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:59.048347 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:59.048357 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:50:59.048397 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:59.048399 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:59.048415 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:59.048537 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:50:59.048743 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:59.049858 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:59.051017 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:59.051746 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:59.052201 (+ 455us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:59.052272 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:59.052296 (+ 24us) server_negotiation.cc:299] Negotiation successful
0806 22:50:59.052338 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":439,"thread_start_us":130,"threads_started":1}
I20260806 22:50:59.052660 30790 tablet_service.cc:1467] Processing CreateTablet for tablet 890cbae6ba004ff69a36745c3a3da6f4 (DEFAULT_TABLE table=test-table [id=95fd8db2a1044fc39f5c19e5c5334eb8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:50:59.053089 30790 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 890cbae6ba004ff69a36745c3a3da6f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:50:59.053099 30901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:59.047891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.66:42075 (local address 127.0.0.1:56112)
0806 22:50:59.048039 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:59.048050 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:59.048103 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:59.048446 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:59.048447 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:59.048449 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:50:59.048523 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:59.048526 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:59.051021 (+ 2495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:59.051021 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:59.051843 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:59.051847 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:59.052792 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:59.052792 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:59.052899 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:59.052904 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:50:59.052961 (+ 57us) client_negotiation.cc:240] Negotiation successful
0806 22:50:59.053024 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"mutex_wait_us":1,"thread_start_us":77,"threads_started":1}
I20260806 22:50:59.053082 30419 tablet_service.cc:1467] Processing CreateTablet for tablet 890cbae6ba004ff69a36745c3a3da6f4 (DEFAULT_TABLE table=test-table [id=95fd8db2a1044fc39f5c19e5c5334eb8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:50:59.053544 30419 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 890cbae6ba004ff69a36745c3a3da6f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:50:59.053623 30905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:59.048053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56112 (local address 127.6.40.66:42075)
0806 22:50:59.048254 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:59.048259 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:59.048270 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:50:59.048314 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:59.048316 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:59.048332 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:59.048605 (+ 273us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:50:59.048695 (+ 90us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:59.049847 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:59.051945 (+ 2098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:59.052631 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:59.052697 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:59.052758 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:59.053463 (+ 705us) server_negotiation.cc:299] Negotiation successful
0806 22:50:59.053509 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":99,"threads_started":1}
I20260806 22:50:59.054314 30607 tablet_service.cc:1467] Processing CreateTablet for tablet 890cbae6ba004ff69a36745c3a3da6f4 (DEFAULT_TABLE table=test-table [id=95fd8db2a1044fc39f5c19e5c5334eb8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:50:59.054656 30607 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 890cbae6ba004ff69a36745c3a3da6f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:50:59.056713 30907 tablet_bootstrap.cc:492] T 890cbae6ba004ff69a36745c3a3da6f4 P 97fac4545d5346d58e255673729c5aa0: Bootstrap starting.
I20260806 22:50:59.057242 30908 tablet_bootstrap.cc:492] T 890cbae6ba004ff69a36745c3a3da6f4 P 4c36bf9d914a4476a33be2dab1bd03f1: Bootstrap starting.
I20260806 22:50:59.057463 30907 tablet_bootstrap.cc:654] T 890cbae6ba004ff69a36745c3a3da6f4 P 97fac4545d5346d58e255673729c5aa0: Neither blocks nor log segments found. Creating new log.
I20260806 22:50:59.057796 30909 tablet_bootstrap.cc:492] T 890cbae6ba004ff69a36745c3a3da6f4 P 95cf1a2e3de74bc590be69c531c3e077: Bootstrap starting.
I20260806 22:50:59.058023 30908 tablet_bootstrap.cc:654] T 890cbae6ba004ff69a36745c3a3da6f4 P 4c36bf9d914a4476a33be2dab1bd03f1: Neither blocks nor log segments found. Creating new log.
I20260806 22:50:59.058138 30907 log.cc:824] T 890cbae6ba004ff69a36745c3a3da6f4 P 97fac4545d5346d58e255673729c5aa0: Log is configured to *not* fsync() on all Append() calls
I20260806 22:50:59.058504 30909 tablet_bootstrap.cc:654] T 890cbae6ba004ff69a36745c3a3da6f4 P 95cf1a2e3de74bc590be69c531c3e077: Neither blocks nor log segments found. Creating new log.
I20260806 22:50:59.058682 30908 log.cc:824] T 890cbae6ba004ff69a36745c3a3da6f4 P 4c36bf9d914a4476a33be2dab1bd03f1: Log is configured to *not* fsync() on all Append() calls
I20260806 22:50:59.059128 30909 log.cc:824] T 890cbae6ba004ff69a36745c3a3da6f4 P 95cf1a2e3de74bc590be69c531c3e077: Log is configured to *not* fsync() on all Append() calls
I20260806 22:50:59.059301 30907 tablet_bootstrap.cc:492] T 890cbae6ba004ff69a36745c3a3da6f4 P 97fac4545d5346d58e255673729c5aa0: No bootstrap required, opened a new log
I20260806 22:50:59.059382 30907 ts_tablet_manager.cc:1397] T 890cbae6ba004ff69a36745c3a3da6f4 P 97fac4545d5346d58e255673729c5aa0: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 22:50:59.059397 30908 tablet_bootstrap.cc:492] T 890cbae6ba004ff69a36745c3a3da6f4 P 4c36bf9d914a4476a33be2dab1bd03f1: No bootstrap required, opened a new log
I20260806 22:50:59.059473 30908 ts_tablet_manager.cc:1397] T 890cbae6ba004ff69a36745c3a3da6f4 P 4c36bf9d914a4476a33be2dab1bd03f1: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:50:59.059958 30909 tablet_bootstrap.cc:492] T 890cbae6ba004ff69a36745c3a3da6f4 P 95cf1a2e3de74bc590be69c531c3e077: No bootstrap required, opened a new log
I20260806 22:50:59.060081 30909 ts_tablet_manager.cc:1397] T 890cbae6ba004ff69a36745c3a3da6f4 P 95cf1a2e3de74bc590be69c531c3e077: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:50:59.061044 30908 raft_consensus.cc:348] T 890cbae6ba004ff69a36745c3a3da6f4 P 4c36bf9d914a4476a33be2dab1bd03f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97fac4545d5346d58e255673729c5aa0" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 42709 } } peers { permanent_uuid: "4c36bf9d914a4476a33be2dab1bd03f1" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 38865 } } peers { permanent_uuid: "95cf1a2e3de74bc590be69c531c3e077" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 42075 } }
I20260806 22:50:59.061184 30908 raft_consensus.cc:374] T 890cbae6ba004ff69a36745c3a3da6f4 P 4c36bf9d914a4476a33be2dab1bd03f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:50:59.061208 30908 raft_consensus.cc:729] T 890cbae6ba004ff69a36745c3a3da6f4 P 4c36bf9d914a4476a33be2dab1bd03f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c36bf9d914a4476a33be2dab1bd03f1, State: Initialized, Role: FOLLOWER
I20260806 22:50:59.061363 30908 consensus_queue.cc:260] T 890cbae6ba004ff69a36745c3a3da6f4 P 4c36bf9d914a4476a33be2dab1bd03f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97fac4545d5346d58e255673729c5aa0" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 42709 } } peers { permanent_uuid: "4c36bf9d914a4476a33be2dab1bd03f1" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 38865 } } peers { permanent_uuid: "95cf1a2e3de74bc590be69c531c3e077" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 42075 } }
I20260806 22:50:59.061633 30908 ts_tablet_manager.cc:1428] T 890cbae6ba004ff69a36745c3a3da6f4 P 4c36bf9d914a4476a33be2dab1bd03f1: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:50:59.061568 30907 raft_consensus.cc:348] T 890cbae6ba004ff69a36745c3a3da6f4 P 97fac4545d5346d58e255673729c5aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97fac4545d5346d58e255673729c5aa0" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 42709 } } peers { permanent_uuid: "4c36bf9d914a4476a33be2dab1bd03f1" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 38865 } } peers { permanent_uuid: "95cf1a2e3de74bc590be69c531c3e077" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 42075 } }
I20260806 22:50:59.061748 30907 raft_consensus.cc:374] T 890cbae6ba004ff69a36745c3a3da6f4 P 97fac4545d5346d58e255673729c5aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:50:59.061769 30516 heartbeater.cc:503] Master 127.6.40.126:39671 was elected leader, sending a full tablet report...
I20260806 22:50:59.061787 30907 raft_consensus.cc:729] T 890cbae6ba004ff69a36745c3a3da6f4 P 97fac4545d5346d58e255673729c5aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97fac4545d5346d58e255673729c5aa0, State: Initialized, Role: FOLLOWER
I20260806 22:50:59.061899 30907 consensus_queue.cc:260] T 890cbae6ba004ff69a36745c3a3da6f4 P 97fac4545d5346d58e255673729c5aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97fac4545d5346d58e255673729c5aa0" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 42709 } } peers { permanent_uuid: "4c36bf9d914a4476a33be2dab1bd03f1" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 38865 } } peers { permanent_uuid: "95cf1a2e3de74bc590be69c531c3e077" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 42075 } }
I20260806 22:50:59.062162 30907 ts_tablet_manager.cc:1428] T 890cbae6ba004ff69a36745c3a3da6f4 P 97fac4545d5346d58e255673729c5aa0: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:50:59.062281 30883 heartbeater.cc:503] Master 127.6.40.126:39671 was elected leader, sending a full tablet report...
I20260806 22:50:59.063091 30909 raft_consensus.cc:348] T 890cbae6ba004ff69a36745c3a3da6f4 P 95cf1a2e3de74bc590be69c531c3e077 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97fac4545d5346d58e255673729c5aa0" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 42709 } } peers { permanent_uuid: "4c36bf9d914a4476a33be2dab1bd03f1" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 38865 } } peers { permanent_uuid: "95cf1a2e3de74bc590be69c531c3e077" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 42075 } }
I20260806 22:50:59.063251 30909 raft_consensus.cc:374] T 890cbae6ba004ff69a36745c3a3da6f4 P 95cf1a2e3de74bc590be69c531c3e077 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:50:59.063294 30909 raft_consensus.cc:729] T 890cbae6ba004ff69a36745c3a3da6f4 P 95cf1a2e3de74bc590be69c531c3e077 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95cf1a2e3de74bc590be69c531c3e077, State: Initialized, Role: FOLLOWER
I20260806 22:50:59.063410 30909 consensus_queue.cc:260] T 890cbae6ba004ff69a36745c3a3da6f4 P 95cf1a2e3de74bc590be69c531c3e077 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97fac4545d5346d58e255673729c5aa0" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 42709 } } peers { permanent_uuid: "4c36bf9d914a4476a33be2dab1bd03f1" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 38865 } } peers { permanent_uuid: "95cf1a2e3de74bc590be69c531c3e077" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 42075 } }
I20260806 22:50:59.063663 30909 ts_tablet_manager.cc:1428] T 890cbae6ba004ff69a36745c3a3da6f4 P 95cf1a2e3de74bc590be69c531c3e077: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260806 22:50:59.063704 30699 heartbeater.cc:503] Master 127.6.40.126:39671 was elected leader, sending a full tablet report...
W20260806 22:50:59.102002 30700 tablet.cc:2369] T 890cbae6ba004ff69a36745c3a3da6f4 P 95cf1a2e3de74bc590be69c531c3e077: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:50:59.113734 30918 raft_consensus.cc:482] T 890cbae6ba004ff69a36745c3a3da6f4 P 95cf1a2e3de74bc590be69c531c3e077 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:50:59.113858 30918 raft_consensus.cc:504] T 890cbae6ba004ff69a36745c3a3da6f4 P 95cf1a2e3de74bc590be69c531c3e077 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97fac4545d5346d58e255673729c5aa0" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 42709 } } peers { permanent_uuid: "4c36bf9d914a4476a33be2dab1bd03f1" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 38865 } } peers { permanent_uuid: "95cf1a2e3de74bc590be69c531c3e077" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 42075 } }
I20260806 22:50:59.114282 30918 leader_election.cc:288] T 890cbae6ba004ff69a36745c3a3da6f4 P 95cf1a2e3de74bc590be69c531c3e077 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97fac4545d5346d58e255673729c5aa0 (127.6.40.67:42709), 4c36bf9d914a4476a33be2dab1bd03f1 (127.6.40.65:38865)
I20260806 22:50:59.118245 30902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:59.114685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.66:39653 (local address 127.6.40.65:38865)
0806 22:50:59.114735 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:59.114737 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:59.114872 (+ 135us) server_negotiation.cc:407] Connection header received
0806 22:50:59.114913 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:59.114914 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:59.114925 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:59.115013 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:50:59.115144 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:59.115828 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:59.117303 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:59.117883 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:59.117961 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:59.117973 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:59.118131 (+ 158us) server_negotiation.cc:299] Negotiation successful
0806 22:50:59.118171 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 22:50:59.118245 30900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:59.114571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.66:34953 (local address 127.6.40.67:42709)
0806 22:50:59.114685 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:59.114688 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:59.114699 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:50:59.114731 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:59.114732 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:59.114746 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:59.114808 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:50:59.114991 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:59.115770 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:59.117303 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:59.117883 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:59.117961 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:59.117972 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:59.118131 (+ 159us) server_negotiation.cc:299] Negotiation successful
0806 22:50:59.118171 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260806 22:50:59.118283 30701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:59.114485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.67:42709 (local address 127.6.40.66:34953)
0806 22:50:59.114556 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:59.114570 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:59.114632 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:59.114818 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:59.114820 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:59.114822 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:50:59.114907 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:59.114910 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:59.115849 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:59.115849 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:59.117217 (+ 1368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:59.117221 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:59.117978 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:59.117978 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:59.118056 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:59.118057 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 22:50:59.118116 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 22:50:59.118166 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44,"mutex_wait_us":60}
I20260806 22:50:59.118291 30921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:59.114603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.65:38865 (local address 127.6.40.66:39653)
0806 22:50:59.114797 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:59.114807 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:59.114863 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:59.115015 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:59.115016 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:59.115017 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:50:59.115082 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:59.115084 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:59.115899 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:59.115900 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:59.117217 (+ 1317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:59.117221 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:59.117972 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:59.117973 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:59.118052 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:59.118057 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:50:59.118116 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 22:50:59.118166 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"mutex_wait_us":15,"thread_start_us":138,"threads_started":1}
I20260806 22:50:59.118723 30450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "890cbae6ba004ff69a36745c3a3da6f4" candidate_uuid: "95cf1a2e3de74bc590be69c531c3e077" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c36bf9d914a4476a33be2dab1bd03f1" is_pre_election: true
I20260806 22:50:59.118688 30816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "890cbae6ba004ff69a36745c3a3da6f4" candidate_uuid: "95cf1a2e3de74bc590be69c531c3e077" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97fac4545d5346d58e255673729c5aa0" is_pre_election: true
I20260806 22:50:59.118878 30450 raft_consensus.cc:2454] T 890cbae6ba004ff69a36745c3a3da6f4 P 4c36bf9d914a4476a33be2dab1bd03f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95cf1a2e3de74bc590be69c531c3e077 in term 0.
I20260806 22:50:59.118878 30816 raft_consensus.cc:2454] T 890cbae6ba004ff69a36745c3a3da6f4 P 97fac4545d5346d58e255673729c5aa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95cf1a2e3de74bc590be69c531c3e077 in term 0.
I20260806 22:50:59.119110 30546 leader_election.cc:302] T 890cbae6ba004ff69a36745c3a3da6f4 P 95cf1a2e3de74bc590be69c531c3e077 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95cf1a2e3de74bc590be69c531c3e077, 97fac4545d5346d58e255673729c5aa0; no voters:
I20260806 22:50:59.119249 30918 raft_consensus.cc:2781] T 890cbae6ba004ff69a36745c3a3da6f4 P 95cf1a2e3de74bc590be69c531c3e077 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:50:59.119313 30918 raft_consensus.cc:482] T 890cbae6ba004ff69a36745c3a3da6f4 P 95cf1a2e3de74bc590be69c531c3e077 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:50:59.119338 30918 raft_consensus.cc:3037] T 890cbae6ba004ff69a36745c3a3da6f4 P 95cf1a2e3de74bc590be69c531c3e077 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:50:59.120245 30918 raft_consensus.cc:504] T 890cbae6ba004ff69a36745c3a3da6f4 P 95cf1a2e3de74bc590be69c531c3e077 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97fac4545d5346d58e255673729c5aa0" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 42709 } } peers { permanent_uuid: "4c36bf9d914a4476a33be2dab1bd03f1" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 38865 } } peers { permanent_uuid: "95cf1a2e3de74bc590be69c531c3e077" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 42075 } }
I20260806 22:50:59.120378 30918 leader_election.cc:288] T 890cbae6ba004ff69a36745c3a3da6f4 P 95cf1a2e3de74bc590be69c531c3e077 [CANDIDATE]: Term 1 election: Requested vote from peers 97fac4545d5346d58e255673729c5aa0 (127.6.40.67:42709), 4c36bf9d914a4476a33be2dab1bd03f1 (127.6.40.65:38865)
I20260806 22:50:59.120575 30450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "890cbae6ba004ff69a36745c3a3da6f4" candidate_uuid: "95cf1a2e3de74bc590be69c531c3e077" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c36bf9d914a4476a33be2dab1bd03f1"
I20260806 22:50:59.120597 30816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "890cbae6ba004ff69a36745c3a3da6f4" candidate_uuid: "95cf1a2e3de74bc590be69c531c3e077" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97fac4545d5346d58e255673729c5aa0"
I20260806 22:50:59.120672 30816 raft_consensus.cc:3037] T 890cbae6ba004ff69a36745c3a3da6f4 P 97fac4545d5346d58e255673729c5aa0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:50:59.120672 30450 raft_consensus.cc:3037] T 890cbae6ba004ff69a36745c3a3da6f4 P 4c36bf9d914a4476a33be2dab1bd03f1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:50:59.121665 30816 raft_consensus.cc:2454] T 890cbae6ba004ff69a36745c3a3da6f4 P 97fac4545d5346d58e255673729c5aa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95cf1a2e3de74bc590be69c531c3e077 in term 1.
I20260806 22:50:59.121665 30450 raft_consensus.cc:2454] T 890cbae6ba004ff69a36745c3a3da6f4 P 4c36bf9d914a4476a33be2dab1bd03f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95cf1a2e3de74bc590be69c531c3e077 in term 1.
I20260806 22:50:59.121845 30546 leader_election.cc:302] T 890cbae6ba004ff69a36745c3a3da6f4 P 95cf1a2e3de74bc590be69c531c3e077 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95cf1a2e3de74bc590be69c531c3e077, 97fac4545d5346d58e255673729c5aa0; no voters:
I20260806 22:50:59.121933 30918 raft_consensus.cc:2781] T 890cbae6ba004ff69a36745c3a3da6f4 P 95cf1a2e3de74bc590be69c531c3e077 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:50:59.122094 30918 raft_consensus.cc:686] T 890cbae6ba004ff69a36745c3a3da6f4 P 95cf1a2e3de74bc590be69c531c3e077 [term 1 LEADER]: Becoming Leader. State: Replica: 95cf1a2e3de74bc590be69c531c3e077, State: Running, Role: LEADER
I20260806 22:50:59.122185 30918 consensus_queue.cc:237] T 890cbae6ba004ff69a36745c3a3da6f4 P 95cf1a2e3de74bc590be69c531c3e077 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97fac4545d5346d58e255673729c5aa0" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 42709 } } peers { permanent_uuid: "4c36bf9d914a4476a33be2dab1bd03f1" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 38865 } } peers { permanent_uuid: "95cf1a2e3de74bc590be69c531c3e077" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 42075 } }
I20260806 22:50:59.123020 30243 catalog_manager.cc:5495] T 890cbae6ba004ff69a36745c3a3da6f4 P 95cf1a2e3de74bc590be69c531c3e077 reported cstate change: term changed from 0 to 1, leader changed from <none> to 95cf1a2e3de74bc590be69c531c3e077 (127.6.40.66). New cstate: current_term: 1 leader_uuid: "95cf1a2e3de74bc590be69c531c3e077" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97fac4545d5346d58e255673729c5aa0" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 42709 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c36bf9d914a4476a33be2dab1bd03f1" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 38865 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95cf1a2e3de74bc590be69c531c3e077" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 42075 } health_report { overall_health: HEALTHY } } }
I20260806 22:50:59.142812 30905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:59.140308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56120 (local address 127.6.40.66:42075)
0806 22:50:59.140373 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:59.140375 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:59.140385 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:50:59.140465 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:59.140466 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:59.140485 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:59.140561 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:50:59.140690 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:59.141237 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:59.142021 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:59.142221 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:59.142294 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:59.142454 (+ 160us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:50:59.142559 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:50:59.142652 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:59.142677 (+ 25us) server_negotiation.cc:299] Negotiation successful
0806 22:50:59.142697 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 22:50:59.144697 30816 raft_consensus.cc:1261] T 890cbae6ba004ff69a36745c3a3da6f4 P 97fac4545d5346d58e255673729c5aa0 [term 1 FOLLOWER]: Refusing update from remote peer 95cf1a2e3de74bc590be69c531c3e077: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:50:59.144734 30450 raft_consensus.cc:1261] T 890cbae6ba004ff69a36745c3a3da6f4 P 4c36bf9d914a4476a33be2dab1bd03f1 [term 1 FOLLOWER]: Refusing update from remote peer 95cf1a2e3de74bc590be69c531c3e077: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:50:59.145071 30918 consensus_queue.cc:1035] T 890cbae6ba004ff69a36745c3a3da6f4 P 95cf1a2e3de74bc590be69c531c3e077 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97fac4545d5346d58e255673729c5aa0" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 42709 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:50:59.145212 30923 consensus_queue.cc:1035] T 890cbae6ba004ff69a36745c3a3da6f4 P 95cf1a2e3de74bc590be69c531c3e077 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c36bf9d914a4476a33be2dab1bd03f1" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 38865 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:50:59.148262 30930 mvcc.cc:205] Tried to move back new op lower bound from 7315688075853578240 to 7315688075764948992. Current Snapshot: MvccSnapshot[applied={T|T < 7315688075853578240 or (T in {7315688075853578240})}]
I20260806 22:50:59.148794 30929 mvcc.cc:205] Tried to move back new op lower bound from 7315688075853578240 to 7315688075764948992. Current Snapshot: MvccSnapshot[applied={T|T < 7315688075853578240}]
I20260806 22:50:59.150084 30246 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53314:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:50:59.150187 30246 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53314:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:50:59.150899 30246 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 21cdf9be9e95482cb228b878ef4e48f6: Sending DeleteTablet for 3 replicas of tablet 890cbae6ba004ff69a36745c3a3da6f4
I20260806 22:50:59.151227 30790 tablet_service.cc:1514] Processing DeleteTablet for tablet 890cbae6ba004ff69a36745c3a3da6f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:50:59 UTC) from {username='kudu', principal='kudu/127.6.40.126@KRBTEST.COM'} at 127.0.0.1:36088
I20260806 22:50:59.151319 30419 tablet_service.cc:1514] Processing DeleteTablet for tablet 890cbae6ba004ff69a36745c3a3da6f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:50:59 UTC) from {username='kudu', principal='kudu/127.6.40.126@KRBTEST.COM'} at 127.0.0.1:56946
I20260806 22:50:59.151582 30939 tablet_replica.cc:330] T 890cbae6ba004ff69a36745c3a3da6f4 P 4c36bf9d914a4476a33be2dab1bd03f1: stopping tablet replica
I20260806 22:50:59.151638 30607 tablet_service.cc:1514] Processing DeleteTablet for tablet 890cbae6ba004ff69a36745c3a3da6f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:50:59 UTC) from {username='kudu', principal='kudu/127.6.40.126@KRBTEST.COM'} at 127.0.0.1:56112
I20260806 22:50:59.151692 30939 raft_consensus.cc:2229] T 890cbae6ba004ff69a36745c3a3da6f4 P 4c36bf9d914a4476a33be2dab1bd03f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:50:59.151788 30939 raft_consensus.cc:2258] T 890cbae6ba004ff69a36745c3a3da6f4 P 4c36bf9d914a4476a33be2dab1bd03f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:50:59.151818 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 30337
I20260806 22:50:59.151983 30940 tablet_replica.cc:330] T 890cbae6ba004ff69a36745c3a3da6f4 P 95cf1a2e3de74bc590be69c531c3e077: stopping tablet replica
I20260806 22:50:59.152123 30940 raft_consensus.cc:2229] T 890cbae6ba004ff69a36745c3a3da6f4 P 95cf1a2e3de74bc590be69c531c3e077 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:50:59.152271 30940 raft_consensus.cc:2258] T 890cbae6ba004ff69a36745c3a3da6f4 P 95cf1a2e3de74bc590be69c531c3e077 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:50:59.155694 30940 ts_tablet_manager.cc:1905] T 890cbae6ba004ff69a36745c3a3da6f4 P 95cf1a2e3de74bc590be69c531c3e077: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:50:59.158239 30940 ts_tablet_manager.cc:1918] T 890cbae6ba004ff69a36745c3a3da6f4 P 95cf1a2e3de74bc590be69c531c3e077: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:50:59.158353 30940 log.cc:1196] T 890cbae6ba004ff69a36745c3a3da6f4 P 95cf1a2e3de74bc590be69c531c3e077: Deleting WAL directory at /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/minicluster-data/ts-1/wal/wals/890cbae6ba004ff69a36745c3a3da6f4
I20260806 22:50:59.158740 30940 ts_tablet_manager.cc:1939] T 890cbae6ba004ff69a36745c3a3da6f4 P 95cf1a2e3de74bc590be69c531c3e077: Deleting consensus metadata
I20260806 22:50:59.159287 30229 catalog_manager.cc:4841] TS 95cf1a2e3de74bc590be69c531c3e077 (127.6.40.66:42075): tablet 890cbae6ba004ff69a36745c3a3da6f4 (table test-table [id=95fd8db2a1044fc39f5c19e5c5334eb8]) successfully deleted
I20260806 22:50:59.160981 30938 tablet_replica.cc:330] T 890cbae6ba004ff69a36745c3a3da6f4 P 97fac4545d5346d58e255673729c5aa0: stopping tablet replica
I20260806 22:50:59.161175 30938 raft_consensus.cc:2229] T 890cbae6ba004ff69a36745c3a3da6f4 P 97fac4545d5346d58e255673729c5aa0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:50:59.161350 30938 raft_consensus.cc:2258] T 890cbae6ba004ff69a36745c3a3da6f4 P 97fac4545d5346d58e255673729c5aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:50:59.164165 30938 ts_tablet_manager.cc:1905] T 890cbae6ba004ff69a36745c3a3da6f4 P 97fac4545d5346d58e255673729c5aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:50:59.166255 30938 ts_tablet_manager.cc:1918] T 890cbae6ba004ff69a36745c3a3da6f4 P 97fac4545d5346d58e255673729c5aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:50:59.166344 30938 log.cc:1196] T 890cbae6ba004ff69a36745c3a3da6f4 P 97fac4545d5346d58e255673729c5aa0: Deleting WAL directory at /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056405724609-6305-0/minicluster-data/ts-2/wal/wals/890cbae6ba004ff69a36745c3a3da6f4
I20260806 22:50:59.166652 30938 ts_tablet_manager.cc:1939] T 890cbae6ba004ff69a36745c3a3da6f4 P 97fac4545d5346d58e255673729c5aa0: Deleting consensus metadata
I20260806 22:50:59.167182 30229 catalog_manager.cc:4841] TS 97fac4545d5346d58e255673729c5aa0 (127.6.40.67:42709): tablet 890cbae6ba004ff69a36745c3a3da6f4 (table test-table [id=95fd8db2a1044fc39f5c19e5c5334eb8]) successfully deleted
W20260806 22:50:59.168987 30229 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.40.65:38865 (error 108)
I20260806 22:50:59.169256 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 30521
I20260806 22:50:59.172747 30901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:59.169388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.65:38865 (local address 127.0.0.1:56960)
0806 22:50:59.172664 (+ 3276us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:59.172713 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.65:38865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3252}
W20260806 22:50:59.172917 30229 catalog_manager.cc:4584] TS 4c36bf9d914a4476a33be2dab1bd03f1 (127.6.40.65:38865): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 890cbae6ba004ff69a36745c3a3da6f4: Network error: Client connection negotiation failed: client connection to 127.6.40.65:38865: connect: Connection refused (error 111)
I20260806 22:50:59.182230 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 30706
I20260806 22:50:59.191005 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 30207
2026-08-06T22:50:59Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2220 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-06T22:50:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:50:59Z Disabled control of system clock
I20260806 22:50:59.227051 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056405724609-6305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056405724609-6305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056405724609-6305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.126:34759
--webserver_interface=127.6.40.126
--webserver_port=0
--builtin_ntp_servers=127.6.40.84:37749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.40.126:34759
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 22:50:59.309303 30949 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:50:59.309530 30949 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:50:59.309551 30949 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:50:59.310663 30949 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:50:59.310698 30949 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:50:59.310710 30949 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:50:59.310724 30949 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:50:59.310735 30949 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:50:59.312002 30949 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:37749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056405724609-6305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056405724609-6305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.40.126:34759
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.126:34759
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30949
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056405724609-6305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:50:59.312192 30949 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:50:59.312426 30949 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:50:59.317312 30954 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:59.317356 30955 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:59.317327 30958 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:59.317512 30949 server_base.cc:1034] running on GCE node
I20260806 22:50:59.317857 30949 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:50:59.318161 30949 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:50:59.319355 30949 hybrid_clock.cc:648] HybridClock initialized: now 1786056659319338 us; error 44 us; skew 500 ppm
I20260806 22:50:59.320715 30949 webserver.cc:460] Webserver started at http://127.6.40.126:34261/ using document root <none> and password file <none>
I20260806 22:50:59.321040 30949 fs_manager.cc:359] Metadata directory not provided
I20260806 22:50:59.321122 30949 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:50:59.321295 30949 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:50:59.322508 30949 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056405724609-6305-0/minicluster-data/master-0/data/instance:
uuid: "8b8567ff38284c8fb7318fb610cd1c6b"
format_stamp: "Formatted at 2026-08-06 22:50:59 on dist-test-slave-zh2d"
server_key: "57cd797d51ceeb8a0815c790de4c2b81"
server_key_iv: "940dcddc7994c3c1d54e97be957d9e6b"
server_key_version: "encryptionkey@0"
I20260806 22:50:59.322917 30949 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056405724609-6305-0/minicluster-data/master-0/wal/instance:
uuid: "8b8567ff38284c8fb7318fb610cd1c6b"
format_stamp: "Formatted at 2026-08-06 22:50:59 on dist-test-slave-zh2d"
server_key: "57cd797d51ceeb8a0815c790de4c2b81"
server_key_iv: "940dcddc7994c3c1d54e97be957d9e6b"
server_key_version: "encryptionkey@0"
I20260806 22:50:59.324430 30949 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:50:59.325294 30972 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:59.325484 30949 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 22:50:59.325595 30949 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056405724609-6305-0/minicluster-data/master-0/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056405724609-6305-0/minicluster-data/master-0/wal
uuid: "8b8567ff38284c8fb7318fb610cd1c6b"
format_stamp: "Formatted at 2026-08-06 22:50:59 on dist-test-slave-zh2d"
server_key: "57cd797d51ceeb8a0815c790de4c2b81"
server_key_iv: "940dcddc7994c3c1d54e97be957d9e6b"
server_key_version: "encryptionkey@0"
I20260806 22:50:59.325668 30949 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056405724609-6305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056405724609-6305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056405724609-6305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:50:59.336148 30949 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:50:59.336467 30949 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:50:59.336613 30949 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:50:59.340763 30949 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.126:34759
I20260806 22:50:59.340799 31033 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.126:34759 every 8 connection(s)
I20260806 22:50:59.341173 30949 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
I20260806 22:50:59.341676 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 30949
I20260806 22:50:59.341796 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056405724609-6305-0/minicluster-data/master-0/wal/instance
I20260806 22:50:59.342078 6305 external_mini_cluster.cc:1428] Setting key 7de753577be4c1a0223fedbaf46601ab
I20260806 22:50:59.342397 31035 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:50:59.345945 31035 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b8567ff38284c8fb7318fb610cd1c6b: Bootstrap starting.
I20260806 22:50:59.346698 31035 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8b8567ff38284c8fb7318fb610cd1c6b: Neither blocks nor log segments found. Creating new log.
I20260806 22:50:59.347061 31035 log.cc:824] T 00000000000000000000000000000000 P 8b8567ff38284c8fb7318fb610cd1c6b: Log is configured to *not* fsync() on all Append() calls
I20260806 22:50:59.347941 31035 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b8567ff38284c8fb7318fb610cd1c6b: No bootstrap required, opened a new log
I20260806 22:50:59.348572 31040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:59.342597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39420 (local address 127.6.40.126:34759)
0806 22:50:59.342876 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:59.342885 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:59.342918 (+ 33us) server_negotiation.cc:407] Connection header received
0806 22:50:59.343605 (+ 687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:59.343615 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:59.343701 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:59.343865 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:59.344028 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:59.345398 (+ 1370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:59.346172 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:59.346770 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:59.346903 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:59.347861 (+ 958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:59.347866 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:59.347866 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:50:59.348092 (+ 226us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:59.348137 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:50:59.348152 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:59.348220 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:59.348275 (+ 55us) server_negotiation.cc:299] Negotiation successful
0806 22:50:59.348353 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":158,"threads_started":1}
I20260806 22:50:59.350188 31035 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8b8567ff38284c8fb7318fb610cd1c6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b8567ff38284c8fb7318fb610cd1c6b" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 34759 } }
I20260806 22:50:59.350318 31035 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8b8567ff38284c8fb7318fb610cd1c6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:50:59.350342 31035 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8b8567ff38284c8fb7318fb610cd1c6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b8567ff38284c8fb7318fb610cd1c6b, State: Initialized, Role: FOLLOWER
I20260806 22:50:59.350487 31035 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8b8567ff38284c8fb7318fb610cd1c6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b8567ff38284c8fb7318fb610cd1c6b" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 34759 } }
I20260806 22:50:59.350559 31035 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8b8567ff38284c8fb7318fb610cd1c6b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:50:59.350600 31035 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8b8567ff38284c8fb7318fb610cd1c6b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:50:59.350647 31035 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8b8567ff38284c8fb7318fb610cd1c6b [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:50:59.351164 31035 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8b8567ff38284c8fb7318fb610cd1c6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b8567ff38284c8fb7318fb610cd1c6b" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 34759 } }
I20260806 22:50:59.351275 31035 leader_election.cc:302] T 00000000000000000000000000000000 P 8b8567ff38284c8fb7318fb610cd1c6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8b8567ff38284c8fb7318fb610cd1c6b; no voters:
I20260806 22:50:59.351445 31035 leader_election.cc:288] T 00000000000000000000000000000000 P 8b8567ff38284c8fb7318fb610cd1c6b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:50:59.351482 31042 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8b8567ff38284c8fb7318fb610cd1c6b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:50:59.351643 31042 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8b8567ff38284c8fb7318fb610cd1c6b [term 1 LEADER]: Becoming Leader. State: Replica: 8b8567ff38284c8fb7318fb610cd1c6b, State: Running, Role: LEADER
I20260806 22:50:59.351727 31035 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8b8567ff38284c8fb7318fb610cd1c6b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:50:59.351737 31042 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8b8567ff38284c8fb7318fb610cd1c6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b8567ff38284c8fb7318fb610cd1c6b" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 34759 } }
I20260806 22:50:59.352195 31044 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b8567ff38284c8fb7318fb610cd1c6b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8b8567ff38284c8fb7318fb610cd1c6b. Latest consensus state: current_term: 1 leader_uuid: "8b8567ff38284c8fb7318fb610cd1c6b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b8567ff38284c8fb7318fb610cd1c6b" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 34759 } } }
I20260806 22:50:59.352293 31044 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b8567ff38284c8fb7318fb610cd1c6b [sys.catalog]: This master's current role is: LEADER
I20260806 22:50:59.352468 31043 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b8567ff38284c8fb7318fb610cd1c6b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8b8567ff38284c8fb7318fb610cd1c6b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b8567ff38284c8fb7318fb610cd1c6b" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 34759 } } }
I20260806 22:50:59.352545 31043 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b8567ff38284c8fb7318fb610cd1c6b [sys.catalog]: This master's current role is: LEADER
I20260806 22:50:59.352509 31054 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:50:59.353133 31054 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:50:59.354580 31054 catalog_manager.cc:1347] Generated new cluster ID: d647b143528549ebad3aa18ea5820115
I20260806 22:50:59.354637 31054 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:50:59.361836 31054 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:50:59.362349 31054 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:50:59.368372 31054 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8b8567ff38284c8fb7318fb610cd1c6b: Generated new TSK 0
I20260806 22:50:59.368567 31054 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:50:59.370342 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056405724609-6305-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056405724609-6305-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056405724609-6305-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.65:0
--local_ip_for_outbound_sockets=127.6.40.65
--webserver_interface=127.6.40.65
--webserver_port=0
--tserver_master_addrs=127.6.40.126:34759
--builtin_ntp_servers=127.6.40.84:37749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 22:50:59.454217 31071 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:50:59.454420 31071 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:50:59.454442 31071 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:50:59.455574 31071 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:50:59.455636 31071 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:50:59.455678 31071 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.65
I20260806 22:50:59.457098 31071 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:37749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056405724609-6305-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056405724609-6305-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.40.65
--webserver_port=0
--tserver_master_addrs=127.6.40.126:34759
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31071
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.65
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056405724609-6305-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:50:59.457351 31071 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:50:59.457654 31071 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:50:59.463688 31085 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:59.463660 31078 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:59.463968 31071 server_base.cc:1034] running on GCE node
W20260806 22:50:59.463653 31081 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:59.464270 31071 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:50:59.464583 31071 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:50:59.465775 31071 hybrid_clock.cc:648] HybridClock initialized: now 1786056659465752 us; error 41 us; skew 500 ppm
I20260806 22:50:59.467125 31071 webserver.cc:460] Webserver started at http://127.6.40.65:37153/ using document root <none> and password file <none>
I20260806 22:50:59.467379 31071 fs_manager.cc:359] Metadata directory not provided
I20260806 22:50:59.467445 31071 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:50:59.467622 31071 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:50:59.468727 31071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056405724609-6305-0/minicluster-data/ts-0/data/instance:
uuid: "74b0f79680204e4fb8f2793666866381"
format_stamp: "Formatted at 2026-08-06 22:50:59 on dist-test-slave-zh2d"
server_key: "575f2788696450900be4c68cf5dfc5be"
server_key_iv: "4b51bb8afe17be062e5c2d2d5d6b1205"
server_key_version: "encryptionkey@0"
I20260806 22:50:59.469159 31071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056405724609-6305-0/minicluster-data/ts-0/wal/instance:
uuid: "74b0f79680204e4fb8f2793666866381"
format_stamp: "Formatted at 2026-08-06 22:50:59 on dist-test-slave-zh2d"
server_key: "575f2788696450900be4c68cf5dfc5be"
server_key_iv: "4b51bb8afe17be062e5c2d2d5d6b1205"
server_key_version: "encryptionkey@0"
I20260806 22:50:59.470633 31071 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:50:59.471433 31093 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:59.471597 31071 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:50:59.471701 31071 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056405724609-6305-0/minicluster-data/ts-0/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056405724609-6305-0/minicluster-data/ts-0/wal
uuid: "74b0f79680204e4fb8f2793666866381"
format_stamp: "Formatted at 2026-08-06 22:50:59 on dist-test-slave-zh2d"
server_key: "575f2788696450900be4c68cf5dfc5be"
server_key_iv: "4b51bb8afe17be062e5c2d2d5d6b1205"
server_key_version: "encryptionkey@0"
I20260806 22:50:59.471776 31071 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056405724609-6305-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056405724609-6305-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056405724609-6305-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:50:59.486999 31071 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:50:59.487339 31071 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:50:59.487497 31071 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:50:59.487746 31071 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:50:59.488087 31071 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:50:59.488121 31071 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:59.488162 31071 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:50:59.488188 31071 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:59.494341 31071 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.65:35591
I20260806 22:50:59.494395 31260 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.65:35591 every 8 connection(s)
I20260806 22:50:59.494745 31071 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
I20260806 22:50:59.496292 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 31071
I20260806 22:50:59.496417 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056405724609-6305-0/minicluster-data/ts-0/wal/instance
I20260806 22:50:59.496675 6305 external_mini_cluster.cc:1428] Setting key 7d750da2434e7aba21ceeca6dff5ef94
I20260806 22:50:59.497669 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056405724609-6305-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056405724609-6305-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056405724609-6305-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.66:0
--local_ip_for_outbound_sockets=127.6.40.66
--webserver_interface=127.6.40.66
--webserver_port=0
--tserver_master_addrs=127.6.40.126:34759
--builtin_ntp_servers=127.6.40.84:37749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 22:50:59.501098 31040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:59.495501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.65:39355 (local address 127.6.40.126:34759)
0806 22:50:59.495566 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:59.495569 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:59.495794 (+ 225us) server_negotiation.cc:407] Connection header received
0806 22:50:59.496537 (+ 743us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:59.496539 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:59.496555 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:59.496627 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:59.497391 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:59.497995 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:59.499522 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:59.499833 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:59.499909 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:59.500805 (+ 896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:59.500806 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:59.500806 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:50:59.500811 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:59.500828 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:50:59.500832 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:59.500920 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:59.501009 (+ 89us) server_negotiation.cc:299] Negotiation successful
0806 22:50:59.501035 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 22:50:59.501400 31264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:59.495487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:34759 (local address 127.6.40.65:39355)
0806 22:50:59.495671 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:59.495706 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:59.496350 (+ 644us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:59.496705 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:59.496710 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:59.496722 (+ 12us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:59.497318 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:59.497324 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:59.498093 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:59.498095 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:59.499412 (+ 1317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:59.499416 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:59.499923 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:59.499924 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:59.500061 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:59.500647 (+ 586us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:50:59.500648 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:59.500687 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:50:59.500718 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:50:59.500931 (+ 213us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:50:59.500937 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:50:59.500940 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:50:59.501006 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 22:50:59.501126 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":77,"threads_started":1}
I20260806 22:50:59.502058 31262 heartbeater.cc:347] Connected to a master server at 127.6.40.126:34759
I20260806 22:50:59.502172 31262 heartbeater.cc:464] Registering TS with master...
I20260806 22:50:59.502364 31262 heartbeater.cc:511] Master 127.6.40.126:34759 requested a full tablet report, sending...
I20260806 22:50:59.502899 30993 ts_manager.cc:194] Registered new tserver with Master: 74b0f79680204e4fb8f2793666866381 (127.6.40.65:35591)
I20260806 22:50:59.503542 30993 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.40.65:39355
W20260806 22:50:59.585124 31265 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:50:59.585320 31265 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:50:59.585341 31265 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:50:59.586570 31265 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:50:59.586612 31265 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:50:59.586658 31265 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.66
I20260806 22:50:59.588004 31265 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:37749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056405724609-6305-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056405724609-6305-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.40.66
--webserver_port=0
--tserver_master_addrs=127.6.40.126:34759
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31265
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.66
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056405724609-6305-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:50:59.588260 31265 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:50:59.588573 31265 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:50:59.593710 31272 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:59.593715 31274 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:59.593720 31271 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:59.594683 31265 server_base.cc:1034] running on GCE node
I20260806 22:50:59.594924 31265 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:50:59.595180 31265 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:50:59.596343 31265 hybrid_clock.cc:648] HybridClock initialized: now 1786056659596330 us; error 44 us; skew 500 ppm
I20260806 22:50:59.597659 31265 webserver.cc:460] Webserver started at http://127.6.40.66:46735/ using document root <none> and password file <none>
I20260806 22:50:59.597930 31265 fs_manager.cc:359] Metadata directory not provided
I20260806 22:50:59.598016 31265 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:50:59.598170 31265 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:50:59.599251 31265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056405724609-6305-0/minicluster-data/ts-1/data/instance:
uuid: "4e4d21ffa4864a8aa57fd9f79a2ea625"
format_stamp: "Formatted at 2026-08-06 22:50:59 on dist-test-slave-zh2d"
server_key: "7684808b75c30c4de2e41922ee63f84f"
server_key_iv: "f0ca1f722039a0c378c73f9d607eb23c"
server_key_version: "encryptionkey@0"
I20260806 22:50:59.599620 31265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056405724609-6305-0/minicluster-data/ts-1/wal/instance:
uuid: "4e4d21ffa4864a8aa57fd9f79a2ea625"
format_stamp: "Formatted at 2026-08-06 22:50:59 on dist-test-slave-zh2d"
server_key: "7684808b75c30c4de2e41922ee63f84f"
server_key_iv: "f0ca1f722039a0c378c73f9d607eb23c"
server_key_version: "encryptionkey@0"
I20260806 22:50:59.601001 31265 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:50:59.601737 31283 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:59.601905 31265 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:50:59.602010 31265 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056405724609-6305-0/minicluster-data/ts-1/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056405724609-6305-0/minicluster-data/ts-1/wal
uuid: "4e4d21ffa4864a8aa57fd9f79a2ea625"
format_stamp: "Formatted at 2026-08-06 22:50:59 on dist-test-slave-zh2d"
server_key: "7684808b75c30c4de2e41922ee63f84f"
server_key_iv: "f0ca1f722039a0c378c73f9d607eb23c"
server_key_version: "encryptionkey@0"
I20260806 22:50:59.602087 31265 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056405724609-6305-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056405724609-6305-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056405724609-6305-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:50:59.619042 31265 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:50:59.619386 31265 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:50:59.619539 31265 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:50:59.619797 31265 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:50:59.620141 31265 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:50:59.620193 31265 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:59.620234 31265 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:50:59.620258 31265 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:59.626551 31265 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.66:37151
I20260806 22:50:59.626598 31455 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.66:37151 every 8 connection(s)
I20260806 22:50:59.626928 31265 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
I20260806 22:50:59.632030 31040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:59.627707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.66:53277 (local address 127.6.40.126:34759)
0806 22:50:59.627774 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:59.627777 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:59.628058 (+ 281us) server_negotiation.cc:407] Connection header received
0806 22:50:59.628536 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:59.628537 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:59.628548 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:59.628604 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:59.629356 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:59.629841 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:59.630771 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:59.630947 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:59.631010 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:59.631812 (+ 802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:59.631812 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:59.631813 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:50:59.631816 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:59.631827 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:50:59.631830 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:59.631868 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:59.631946 (+ 78us) server_negotiation.cc:299] Negotiation successful
0806 22:50:59.631964 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 22:50:59.632282 31458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:59.627736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:34759 (local address 127.6.40.66:53277)
0806 22:50:59.627925 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:59.627969 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:59.628452 (+ 483us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:59.628696 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:59.628700 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:59.628711 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:59.629281 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:59.629286 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:59.629919 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:59.629919 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:59.630703 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:59.630705 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:59.631019 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:59.631020 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:59.631125 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:59.631692 (+ 567us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:50:59.631693 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:59.631722 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:50:59.631744 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:50:59.631884 (+ 140us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:50:59.631886 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:50:59.631889 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:50:59.631937 (+ 48us) client_negotiation.cc:240] Negotiation successful
0806 22:50:59.632020 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":87,"threads_started":1}
I20260806 22:50:59.632786 31456 heartbeater.cc:347] Connected to a master server at 127.6.40.126:34759
I20260806 22:50:59.632916 31456 heartbeater.cc:464] Registering TS with master...
I20260806 22:50:59.633126 31456 heartbeater.cc:511] Master 127.6.40.126:34759 requested a full tablet report, sending...
I20260806 22:50:59.633515 30993 ts_manager.cc:194] Registered new tserver with Master: 4e4d21ffa4864a8aa57fd9f79a2ea625 (127.6.40.66:37151)
I20260806 22:50:59.633846 30993 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.40.66:53277
I20260806 22:50:59.634435 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 31265
I20260806 22:50:59.634567 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056405724609-6305-0/minicluster-data/ts-1/wal/instance
I20260806 22:50:59.634809 6305 external_mini_cluster.cc:1428] Setting key 5caeaaa15fe92667c8ce3308c449d265
I20260806 22:50:59.635661 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056405724609-6305-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056405724609-6305-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056405724609-6305-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.67:0
--local_ip_for_outbound_sockets=127.6.40.67
--webserver_interface=127.6.40.67
--webserver_port=0
--tserver_master_addrs=127.6.40.126:34759
--builtin_ntp_servers=127.6.40.84:37749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 22:50:59.722864 31459 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:50:59.723068 31459 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:50:59.723090 31459 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:50:59.724222 31459 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:50:59.724263 31459 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:50:59.724313 31459 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.67
I20260806 22:50:59.725687 31459 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:37749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056405724609-6305-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056405724609-6305-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.40.67
--webserver_port=0
--tserver_master_addrs=127.6.40.126:34759
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31459
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.67
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056405724609-6305-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:50:59.725934 31459 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:50:59.726239 31459 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:50:59.731559 31468 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:59.731536 31472 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:59.731647 31459 server_base.cc:1034] running on GCE node
W20260806 22:50:59.731591 31469 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:59.731910 31459 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:50:59.732151 31459 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:50:59.733319 31459 hybrid_clock.cc:648] HybridClock initialized: now 1786056659733308 us; error 48 us; skew 500 ppm
I20260806 22:50:59.734596 31459 webserver.cc:460] Webserver started at http://127.6.40.67:41349/ using document root <none> and password file <none>
I20260806 22:50:59.734831 31459 fs_manager.cc:359] Metadata directory not provided
I20260806 22:50:59.734910 31459 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:50:59.735055 31459 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:50:59.736119 31459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056405724609-6305-0/minicluster-data/ts-2/data/instance:
uuid: "6cf8ac809531456abd8a10183aec1a6c"
format_stamp: "Formatted at 2026-08-06 22:50:59 on dist-test-slave-zh2d"
server_key: "506d3f1673dbb22dfe9452ad35958d4f"
server_key_iv: "89f22eee7c82bf3d9b3d73a29c2c990b"
server_key_version: "encryptionkey@0"
I20260806 22:50:59.736488 31459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056405724609-6305-0/minicluster-data/ts-2/wal/instance:
uuid: "6cf8ac809531456abd8a10183aec1a6c"
format_stamp: "Formatted at 2026-08-06 22:50:59 on dist-test-slave-zh2d"
server_key: "506d3f1673dbb22dfe9452ad35958d4f"
server_key_iv: "89f22eee7c82bf3d9b3d73a29c2c990b"
server_key_version: "encryptionkey@0"
I20260806 22:50:59.737897 31459 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:50:59.738590 31481 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:59.738759 31459 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:50:59.738866 31459 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056405724609-6305-0/minicluster-data/ts-2/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056405724609-6305-0/minicluster-data/ts-2/wal
uuid: "6cf8ac809531456abd8a10183aec1a6c"
format_stamp: "Formatted at 2026-08-06 22:50:59 on dist-test-slave-zh2d"
server_key: "506d3f1673dbb22dfe9452ad35958d4f"
server_key_iv: "89f22eee7c82bf3d9b3d73a29c2c990b"
server_key_version: "encryptionkey@0"
I20260806 22:50:59.738938 31459 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056405724609-6305-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056405724609-6305-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056405724609-6305-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:50:59.752944 31459 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:50:59.753298 31459 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:50:59.753454 31459 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:50:59.753692 31459 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:50:59.754021 31459 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:50:59.754070 31459 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:59.754105 31459 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:50:59.754132 31459 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:59.760035 31459 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.67:37267
I20260806 22:50:59.760113 31645 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.67:37267 every 8 connection(s)
I20260806 22:50:59.760445 31459 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
I20260806 22:50:59.761196 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 31459
I20260806 22:50:59.761332 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056405724609-6305-0/minicluster-data/ts-2/wal/instance
I20260806 22:50:59.761634 6305 external_mini_cluster.cc:1428] Setting key 7a47153c59f19807d4be78871fbfa765
I20260806 22:50:59.765841 31040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:59.761320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.67:59977 (local address 127.6.40.126:34759)
0806 22:50:59.761375 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:59.761377 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:59.761651 (+ 274us) server_negotiation.cc:407] Connection header received
0806 22:50:59.762187 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:59.762188 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:59.762200 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:59.762243 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:59.762991 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:59.763484 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:59.764463 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:59.764640 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:59.764692 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:59.765608 (+ 916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:59.765609 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:59.765609 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:50:59.765614 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:59.765628 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:50:59.765631 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:59.765676 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:59.765761 (+ 85us) server_negotiation.cc:299] Negotiation successful
0806 22:50:59.765783 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 22:50:59.766117 31650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:59.761280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:34759 (local address 127.6.40.67:59977)
0806 22:50:59.761517 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:59.761553 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:59.762106 (+ 553us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:59.762328 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:59.762331 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:59.762339 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:59.762915 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:59.762921 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:59.763569 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:59.763570 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:59.764375 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:59.764377 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:59.764743 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:59.764744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:59.764938 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:59.765467 (+ 529us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:50:59.765467 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:59.765497 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:50:59.765521 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:50:59.765691 (+ 170us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:50:59.765696 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:50:59.765698 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 22:50:59.765756 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 22:50:59.765857 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":114,"threads_started":1}
I20260806 22:50:59.766628 31648 heartbeater.cc:347] Connected to a master server at 127.6.40.126:34759
I20260806 22:50:59.766716 31648 heartbeater.cc:464] Registering TS with master...
I20260806 22:50:59.766896 31648 heartbeater.cc:511] Master 127.6.40.126:34759 requested a full tablet report, sending...
I20260806 22:50:59.767256 30993 ts_manager.cc:194] Registered new tserver with Master: 6cf8ac809531456abd8a10183aec1a6c (127.6.40.67:37267)
I20260806 22:50:59.767597 30993 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.40.67:59977
I20260806 22:50:59.773480 6305 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:50:59.777715 31040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:59.774870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39424 (local address 127.6.40.126:34759)
0806 22:50:59.774927 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:59.774929 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:59.775102 (+ 173us) server_negotiation.cc:407] Connection header received
0806 22:50:59.775145 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:59.775145 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:59.775154 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:59.775197 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:59.775329 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:59.775798 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:59.776505 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:59.776675 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:59.776745 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:59.777489 (+ 744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:59.777490 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:59.777491 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:50:59.777495 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:59.777507 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:50:59.777511 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:59.777554 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:59.777603 (+ 49us) server_negotiation.cc:299] Negotiation successful
0806 22:50:59.777621 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 22:50:59.779350 30993 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39424:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:50:59.780943 30993 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:50:59.786856 31660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:59.783155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34530 (local address 127.6.40.67:37267)
0806 22:50:59.783507 (+ 352us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:59.783511 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:59.783535 (+ 24us) server_negotiation.cc:407] Connection header received
0806 22:50:59.783580 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:59.783582 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:59.783599 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:59.783679 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:50:59.783868 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:59.784681 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:59.785774 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:59.786449 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:59.786537 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:59.786626 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:59.786738 (+ 112us) server_negotiation.cc:299] Negotiation successful
0806 22:50:59.786773 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":73,"threads_started":1}
I20260806 22:50:59.786862 31658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:59.782932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.67:37267 (local address 127.0.0.1:34530)
0806 22:50:59.783139 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:59.783179 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:59.783242 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:59.783754 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:59.783755 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:59.783757 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:50:59.783806 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:59.783808 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:59.784772 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:59.784773 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:59.785678 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:59.785681 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:59.786546 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:59.786547 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:59.786629 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:59.786643 (+ 14us) client_negotiation.cc:769] Sending connection context
0806 22:50:59.786726 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 22:50:59.786777 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"mutex_wait_us":68,"thread_start_us":110,"threads_started":1}
I20260806 22:50:59.787143 31657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:59.782975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.66:37151 (local address 127.0.0.1:59486)
0806 22:50:59.783299 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:59.783310 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:59.783374 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:59.783672 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:59.783675 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:59.783677 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:50:59.783738 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:59.783739 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:59.784772 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:59.784774 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:59.786047 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:59.786050 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:59.786946 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:59.786947 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:59.787027 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:59.787031 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:50:59.787061 (+ 30us) client_negotiation.cc:240] Negotiation successful
0806 22:50:59.787099 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":94,"threads_started":1}
I20260806 22:50:59.787559 31659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:59.783096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59486 (local address 127.6.40.66:37151)
0806 22:50:59.783473 (+ 377us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:59.783479 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:59.783492 (+ 13us) server_negotiation.cc:407] Connection header received
0806 22:50:59.783535 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:59.783537 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:59.783553 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:59.783635 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:50:59.783916 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:59.784679 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:59.786111 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:59.786841 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:59.787333 (+ 492us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:59.787432 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:59.787462 (+ 30us) server_negotiation.cc:299] Negotiation successful
0806 22:50:59.787495 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":94,"threads_started":1}
I20260806 22:50:59.787709 31549 tablet_service.cc:1467] Processing CreateTablet for tablet acc3eeb66c5a43c0bcd31feabd986700 (DEFAULT_TABLE table=test-table [id=07501f8d27cd46ce8e57877a250498ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:50:59.788105 31549 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet acc3eeb66c5a43c0bcd31feabd986700. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:50:59.788203 31365 tablet_service.cc:1467] Processing CreateTablet for tablet acc3eeb66c5a43c0bcd31feabd986700 (DEFAULT_TABLE table=test-table [id=07501f8d27cd46ce8e57877a250498ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:50:59.788491 31662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:59.783409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49642 (local address 127.6.40.65:35591)
0806 22:50:59.783617 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:59.783624 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:59.783634 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:50:59.783672 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:59.783674 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:59.783688 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:59.783758 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:50:59.784796 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:59.785950 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:59.787325 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:59.788030 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:59.788106 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:59.788192 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:59.788357 (+ 165us) server_negotiation.cc:299] Negotiation successful
0806 22:50:59.788401 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":109,"threads_started":1}
I20260806 22:50:59.788601 31365 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet acc3eeb66c5a43c0bcd31feabd986700. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:50:59.788513 31661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:59.783199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.65:35591 (local address 127.0.0.1:49642)
0806 22:50:59.783334 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:59.783343 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:59.783385 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:59.783768 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:59.783769 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:59.783771 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:50:59.783820 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:59.783822 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:59.786043 (+ 2221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:59.786044 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:59.787224 (+ 1180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:59.787228 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:59.788119 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:59.788120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:59.788258 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:59.788263 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:50:59.788329 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 22:50:59.788443 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":80,"threads_started":1}
I20260806 22:50:59.789516 31160 tablet_service.cc:1467] Processing CreateTablet for tablet acc3eeb66c5a43c0bcd31feabd986700 (DEFAULT_TABLE table=test-table [id=07501f8d27cd46ce8e57877a250498ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:50:59.789847 31160 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet acc3eeb66c5a43c0bcd31feabd986700. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:50:59.791131 31663 tablet_bootstrap.cc:492] T acc3eeb66c5a43c0bcd31feabd986700 P 6cf8ac809531456abd8a10183aec1a6c: Bootstrap starting.
I20260806 22:50:59.791743 31663 tablet_bootstrap.cc:654] T acc3eeb66c5a43c0bcd31feabd986700 P 6cf8ac809531456abd8a10183aec1a6c: Neither blocks nor log segments found. Creating new log.
I20260806 22:50:59.792045 31663 log.cc:824] T acc3eeb66c5a43c0bcd31feabd986700 P 6cf8ac809531456abd8a10183aec1a6c: Log is configured to *not* fsync() on all Append() calls
I20260806 22:50:59.792122 31665 tablet_bootstrap.cc:492] T acc3eeb66c5a43c0bcd31feabd986700 P 4e4d21ffa4864a8aa57fd9f79a2ea625: Bootstrap starting.
I20260806 22:50:59.792795 31663 tablet_bootstrap.cc:492] T acc3eeb66c5a43c0bcd31feabd986700 P 6cf8ac809531456abd8a10183aec1a6c: No bootstrap required, opened a new log
I20260806 22:50:59.792826 31665 tablet_bootstrap.cc:654] T acc3eeb66c5a43c0bcd31feabd986700 P 4e4d21ffa4864a8aa57fd9f79a2ea625: Neither blocks nor log segments found. Creating new log.
I20260806 22:50:59.792886 31663 ts_tablet_manager.cc:1397] T acc3eeb66c5a43c0bcd31feabd986700 P 6cf8ac809531456abd8a10183aec1a6c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:50:59.793228 31665 log.cc:824] T acc3eeb66c5a43c0bcd31feabd986700 P 4e4d21ffa4864a8aa57fd9f79a2ea625: Log is configured to *not* fsync() on all Append() calls
I20260806 22:50:59.793206 31675 tablet_bootstrap.cc:492] T acc3eeb66c5a43c0bcd31feabd986700 P 74b0f79680204e4fb8f2793666866381: Bootstrap starting.
I20260806 22:50:59.793911 31675 tablet_bootstrap.cc:654] T acc3eeb66c5a43c0bcd31feabd986700 P 74b0f79680204e4fb8f2793666866381: Neither blocks nor log segments found. Creating new log.
I20260806 22:50:59.794064 31665 tablet_bootstrap.cc:492] T acc3eeb66c5a43c0bcd31feabd986700 P 4e4d21ffa4864a8aa57fd9f79a2ea625: No bootstrap required, opened a new log
I20260806 22:50:59.794140 31665 ts_tablet_manager.cc:1397] T acc3eeb66c5a43c0bcd31feabd986700 P 4e4d21ffa4864a8aa57fd9f79a2ea625: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 22:50:59.794262 31675 log.cc:824] T acc3eeb66c5a43c0bcd31feabd986700 P 74b0f79680204e4fb8f2793666866381: Log is configured to *not* fsync() on all Append() calls
I20260806 22:50:59.795400 31675 tablet_bootstrap.cc:492] T acc3eeb66c5a43c0bcd31feabd986700 P 74b0f79680204e4fb8f2793666866381: No bootstrap required, opened a new log
I20260806 22:50:59.795352 31663 raft_consensus.cc:348] T acc3eeb66c5a43c0bcd31feabd986700 P 6cf8ac809531456abd8a10183aec1a6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cf8ac809531456abd8a10183aec1a6c" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 37267 } } peers { permanent_uuid: "4e4d21ffa4864a8aa57fd9f79a2ea625" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 37151 } } peers { permanent_uuid: "74b0f79680204e4fb8f2793666866381" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 35591 } }
I20260806 22:50:59.795485 31675 ts_tablet_manager.cc:1397] T acc3eeb66c5a43c0bcd31feabd986700 P 74b0f79680204e4fb8f2793666866381: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:50:59.795504 31663 raft_consensus.cc:374] T acc3eeb66c5a43c0bcd31feabd986700 P 6cf8ac809531456abd8a10183aec1a6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:50:59.795542 31663 raft_consensus.cc:729] T acc3eeb66c5a43c0bcd31feabd986700 P 6cf8ac809531456abd8a10183aec1a6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cf8ac809531456abd8a10183aec1a6c, State: Initialized, Role: FOLLOWER
I20260806 22:50:59.795692 31663 consensus_queue.cc:260] T acc3eeb66c5a43c0bcd31feabd986700 P 6cf8ac809531456abd8a10183aec1a6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cf8ac809531456abd8a10183aec1a6c" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 37267 } } peers { permanent_uuid: "4e4d21ffa4864a8aa57fd9f79a2ea625" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 37151 } } peers { permanent_uuid: "74b0f79680204e4fb8f2793666866381" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 35591 } }
I20260806 22:50:59.795976 31663 ts_tablet_manager.cc:1428] T acc3eeb66c5a43c0bcd31feabd986700 P 6cf8ac809531456abd8a10183aec1a6c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:50:59.795987 31648 heartbeater.cc:503] Master 127.6.40.126:34759 was elected leader, sending a full tablet report...
I20260806 22:50:59.796814 31665 raft_consensus.cc:348] T acc3eeb66c5a43c0bcd31feabd986700 P 4e4d21ffa4864a8aa57fd9f79a2ea625 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cf8ac809531456abd8a10183aec1a6c" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 37267 } } peers { permanent_uuid: "4e4d21ffa4864a8aa57fd9f79a2ea625" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 37151 } } peers { permanent_uuid: "74b0f79680204e4fb8f2793666866381" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 35591 } }
I20260806 22:50:59.797039 31665 raft_consensus.cc:374] T acc3eeb66c5a43c0bcd31feabd986700 P 4e4d21ffa4864a8aa57fd9f79a2ea625 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:50:59.797086 31665 raft_consensus.cc:729] T acc3eeb66c5a43c0bcd31feabd986700 P 4e4d21ffa4864a8aa57fd9f79a2ea625 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e4d21ffa4864a8aa57fd9f79a2ea625, State: Initialized, Role: FOLLOWER
I20260806 22:50:59.797194 31665 consensus_queue.cc:260] T acc3eeb66c5a43c0bcd31feabd986700 P 4e4d21ffa4864a8aa57fd9f79a2ea625 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cf8ac809531456abd8a10183aec1a6c" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 37267 } } peers { permanent_uuid: "4e4d21ffa4864a8aa57fd9f79a2ea625" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 37151 } } peers { permanent_uuid: "74b0f79680204e4fb8f2793666866381" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 35591 } }
I20260806 22:50:59.797456 31665 ts_tablet_manager.cc:1428] T acc3eeb66c5a43c0bcd31feabd986700 P 4e4d21ffa4864a8aa57fd9f79a2ea625: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260806 22:50:59.797506 31456 heartbeater.cc:503] Master 127.6.40.126:34759 was elected leader, sending a full tablet report...
I20260806 22:50:59.798053 31675 raft_consensus.cc:348] T acc3eeb66c5a43c0bcd31feabd986700 P 74b0f79680204e4fb8f2793666866381 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cf8ac809531456abd8a10183aec1a6c" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 37267 } } peers { permanent_uuid: "4e4d21ffa4864a8aa57fd9f79a2ea625" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 37151 } } peers { permanent_uuid: "74b0f79680204e4fb8f2793666866381" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 35591 } }
I20260806 22:50:59.798167 31675 raft_consensus.cc:374] T acc3eeb66c5a43c0bcd31feabd986700 P 74b0f79680204e4fb8f2793666866381 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:50:59.798206 31675 raft_consensus.cc:729] T acc3eeb66c5a43c0bcd31feabd986700 P 74b0f79680204e4fb8f2793666866381 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74b0f79680204e4fb8f2793666866381, State: Initialized, Role: FOLLOWER
I20260806 22:50:59.798322 31675 consensus_queue.cc:260] T acc3eeb66c5a43c0bcd31feabd986700 P 74b0f79680204e4fb8f2793666866381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cf8ac809531456abd8a10183aec1a6c" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 37267 } } peers { permanent_uuid: "4e4d21ffa4864a8aa57fd9f79a2ea625" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 37151 } } peers { permanent_uuid: "74b0f79680204e4fb8f2793666866381" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 35591 } }
I20260806 22:50:59.798611 31675 ts_tablet_manager.cc:1428] T acc3eeb66c5a43c0bcd31feabd986700 P 74b0f79680204e4fb8f2793666866381: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:50:59.798753 31262 heartbeater.cc:503] Master 127.6.40.126:34759 was elected leader, sending a full tablet report...
I20260806 22:50:59.852694 31681 raft_consensus.cc:482] T acc3eeb66c5a43c0bcd31feabd986700 P 4e4d21ffa4864a8aa57fd9f79a2ea625 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:50:59.852911 31681 raft_consensus.cc:504] T acc3eeb66c5a43c0bcd31feabd986700 P 4e4d21ffa4864a8aa57fd9f79a2ea625 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cf8ac809531456abd8a10183aec1a6c" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 37267 } } peers { permanent_uuid: "4e4d21ffa4864a8aa57fd9f79a2ea625" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 37151 } } peers { permanent_uuid: "74b0f79680204e4fb8f2793666866381" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 35591 } }
I20260806 22:50:59.853318 31681 leader_election.cc:288] T acc3eeb66c5a43c0bcd31feabd986700 P 4e4d21ffa4864a8aa57fd9f79a2ea625 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6cf8ac809531456abd8a10183aec1a6c (127.6.40.67:37267), 74b0f79680204e4fb8f2793666866381 (127.6.40.65:35591)
I20260806 22:50:59.856092 31660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:59.853640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.66:35909 (local address 127.6.40.67:37267)
0806 22:50:59.853691 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:59.853693 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:59.853707 (+ 14us) server_negotiation.cc:407] Connection header received
0806 22:50:59.853745 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:59.853745 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:59.853755 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:59.853818 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:50:59.853979 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:59.854448 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:59.855376 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:59.855746 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:59.855828 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:59.855839 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:59.855974 (+ 135us) server_negotiation.cc:299] Negotiation successful
0806 22:50:59.856017 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 22:50:59.856091 31458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:59.853519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.67:37267 (local address 127.6.40.66:35909)
0806 22:50:59.853580 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:59.853593 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:59.853680 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:59.853823 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:59.853824 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:59.853826 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:50:59.853900 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:59.853902 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:59.854536 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:59.854538 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:59.855283 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:59.855285 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:59.855827 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:59.855828 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:59.855900 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:59.855903 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:50:59.855967 (+ 64us) client_negotiation.cc:240] Negotiation successful
0806 22:50:59.856017 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260806 22:50:59.856458 31683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:59.853637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.65:35591 (local address 127.6.40.66:35307)
0806 22:50:59.853800 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:59.853809 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:59.853856 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:59.854011 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:59.854012 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:59.854013 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:50:59.854093 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:59.854095 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:59.854731 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:59.854732 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:59.855472 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:59.855473 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:59.856031 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:59.856032 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:59.856100 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:59.856103 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:50:59.856311 (+ 208us) client_negotiation.cc:240] Negotiation successful
0806 22:50:59.856353 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":102,"threads_started":1}
I20260806 22:50:59.856456 31662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:59.853744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.66:35307 (local address 127.6.40.65:35591)
0806 22:50:59.853793 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:59.853795 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:59.853849 (+ 54us) server_negotiation.cc:407] Connection header received
0806 22:50:59.853913 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:59.853914 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:59.853925 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:59.854007 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:50:59.854171 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:59.854650 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:59.855562 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:59.855920 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:59.856015 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:59.856026 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:59.856343 (+ 317us) server_negotiation.cc:299] Negotiation successful
0806 22:50:59.856394 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 22:50:59.856464 31578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acc3eeb66c5a43c0bcd31feabd986700" candidate_uuid: "4e4d21ffa4864a8aa57fd9f79a2ea625" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cf8ac809531456abd8a10183aec1a6c" is_pre_election: true
I20260806 22:50:59.856634 31578 raft_consensus.cc:2454] T acc3eeb66c5a43c0bcd31feabd986700 P 6cf8ac809531456abd8a10183aec1a6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e4d21ffa4864a8aa57fd9f79a2ea625 in term 0.
I20260806 22:50:59.856832 31193 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acc3eeb66c5a43c0bcd31feabd986700" candidate_uuid: "4e4d21ffa4864a8aa57fd9f79a2ea625" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74b0f79680204e4fb8f2793666866381" is_pre_election: true
I20260806 22:50:59.856899 31291 leader_election.cc:302] T acc3eeb66c5a43c0bcd31feabd986700 P 4e4d21ffa4864a8aa57fd9f79a2ea625 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e4d21ffa4864a8aa57fd9f79a2ea625, 6cf8ac809531456abd8a10183aec1a6c; no voters:
I20260806 22:50:59.857028 31193 raft_consensus.cc:2454] T acc3eeb66c5a43c0bcd31feabd986700 P 74b0f79680204e4fb8f2793666866381 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e4d21ffa4864a8aa57fd9f79a2ea625 in term 0.
I20260806 22:50:59.857029 31681 raft_consensus.cc:2781] T acc3eeb66c5a43c0bcd31feabd986700 P 4e4d21ffa4864a8aa57fd9f79a2ea625 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:50:59.857089 31681 raft_consensus.cc:482] T acc3eeb66c5a43c0bcd31feabd986700 P 4e4d21ffa4864a8aa57fd9f79a2ea625 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:50:59.857113 31681 raft_consensus.cc:3037] T acc3eeb66c5a43c0bcd31feabd986700 P 4e4d21ffa4864a8aa57fd9f79a2ea625 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:50:59.857872 31681 raft_consensus.cc:504] T acc3eeb66c5a43c0bcd31feabd986700 P 4e4d21ffa4864a8aa57fd9f79a2ea625 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cf8ac809531456abd8a10183aec1a6c" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 37267 } } peers { permanent_uuid: "4e4d21ffa4864a8aa57fd9f79a2ea625" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 37151 } } peers { permanent_uuid: "74b0f79680204e4fb8f2793666866381" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 35591 } }
I20260806 22:50:59.858006 31681 leader_election.cc:288] T acc3eeb66c5a43c0bcd31feabd986700 P 4e4d21ffa4864a8aa57fd9f79a2ea625 [CANDIDATE]: Term 1 election: Requested vote from peers 6cf8ac809531456abd8a10183aec1a6c (127.6.40.67:37267), 74b0f79680204e4fb8f2793666866381 (127.6.40.65:35591)
I20260806 22:50:59.858194 31193 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acc3eeb66c5a43c0bcd31feabd986700" candidate_uuid: "4e4d21ffa4864a8aa57fd9f79a2ea625" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74b0f79680204e4fb8f2793666866381"
I20260806 22:50:59.858217 31578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acc3eeb66c5a43c0bcd31feabd986700" candidate_uuid: "4e4d21ffa4864a8aa57fd9f79a2ea625" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cf8ac809531456abd8a10183aec1a6c"
I20260806 22:50:59.858268 31193 raft_consensus.cc:3037] T acc3eeb66c5a43c0bcd31feabd986700 P 74b0f79680204e4fb8f2793666866381 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:50:59.858302 31578 raft_consensus.cc:3037] T acc3eeb66c5a43c0bcd31feabd986700 P 6cf8ac809531456abd8a10183aec1a6c [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:50:59.859006 31193 raft_consensus.cc:2454] T acc3eeb66c5a43c0bcd31feabd986700 P 74b0f79680204e4fb8f2793666866381 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e4d21ffa4864a8aa57fd9f79a2ea625 in term 1.
I20260806 22:50:59.859130 31578 raft_consensus.cc:2454] T acc3eeb66c5a43c0bcd31feabd986700 P 6cf8ac809531456abd8a10183aec1a6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e4d21ffa4864a8aa57fd9f79a2ea625 in term 1.
I20260806 22:50:59.859304 31291 leader_election.cc:302] T acc3eeb66c5a43c0bcd31feabd986700 P 4e4d21ffa4864a8aa57fd9f79a2ea625 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e4d21ffa4864a8aa57fd9f79a2ea625, 6cf8ac809531456abd8a10183aec1a6c; no voters:
I20260806 22:50:59.859419 31681 raft_consensus.cc:2781] T acc3eeb66c5a43c0bcd31feabd986700 P 4e4d21ffa4864a8aa57fd9f79a2ea625 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:50:59.859742 31681 raft_consensus.cc:686] T acc3eeb66c5a43c0bcd31feabd986700 P 4e4d21ffa4864a8aa57fd9f79a2ea625 [term 1 LEADER]: Becoming Leader. State: Replica: 4e4d21ffa4864a8aa57fd9f79a2ea625, State: Running, Role: LEADER
I20260806 22:50:59.859864 31681 consensus_queue.cc:237] T acc3eeb66c5a43c0bcd31feabd986700 P 4e4d21ffa4864a8aa57fd9f79a2ea625 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cf8ac809531456abd8a10183aec1a6c" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 37267 } } peers { permanent_uuid: "4e4d21ffa4864a8aa57fd9f79a2ea625" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 37151 } } peers { permanent_uuid: "74b0f79680204e4fb8f2793666866381" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 35591 } }
I20260806 22:50:59.860723 30993 catalog_manager.cc:5495] T acc3eeb66c5a43c0bcd31feabd986700 P 4e4d21ffa4864a8aa57fd9f79a2ea625 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4e4d21ffa4864a8aa57fd9f79a2ea625 (127.6.40.66). New cstate: current_term: 1 leader_uuid: "4e4d21ffa4864a8aa57fd9f79a2ea625" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cf8ac809531456abd8a10183aec1a6c" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 37267 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e4d21ffa4864a8aa57fd9f79a2ea625" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 37151 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74b0f79680204e4fb8f2793666866381" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 35591 } health_report { overall_health: UNKNOWN } } }
W20260806 22:50:59.877457 31457 tablet.cc:2369] T acc3eeb66c5a43c0bcd31feabd986700 P 4e4d21ffa4864a8aa57fd9f79a2ea625: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:50:59.882897 31659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:59.880196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59490 (local address 127.6.40.66:37151)
0806 22:50:59.880256 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:59.880258 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:59.880268 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:50:59.880310 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:59.880311 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:59.880328 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:59.880387 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:50:59.880508 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:59.881082 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:59.882219 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:59.882405 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:59.882455 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:59.882572 (+ 117us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:50:59.882668 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:50:59.882725 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:59.882755 (+ 30us) server_negotiation.cc:299] Negotiation successful
0806 22:50:59.882772 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 22:50:59.884714 31193 raft_consensus.cc:1261] T acc3eeb66c5a43c0bcd31feabd986700 P 74b0f79680204e4fb8f2793666866381 [term 1 FOLLOWER]: Refusing update from remote peer 4e4d21ffa4864a8aa57fd9f79a2ea625: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:50:59.884716 31578 raft_consensus.cc:1261] T acc3eeb66c5a43c0bcd31feabd986700 P 6cf8ac809531456abd8a10183aec1a6c [term 1 FOLLOWER]: Refusing update from remote peer 4e4d21ffa4864a8aa57fd9f79a2ea625: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:50:59.885149 31684 consensus_queue.cc:1035] T acc3eeb66c5a43c0bcd31feabd986700 P 4e4d21ffa4864a8aa57fd9f79a2ea625 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74b0f79680204e4fb8f2793666866381" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 35591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:50:59.885282 31681 consensus_queue.cc:1035] T acc3eeb66c5a43c0bcd31feabd986700 P 4e4d21ffa4864a8aa57fd9f79a2ea625 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cf8ac809531456abd8a10183aec1a6c" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 37267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:50:59.888358 31691 mvcc.cc:205] Tried to move back new op lower bound from 7315688078884847616 to 7315688078786748416. Current Snapshot: MvccSnapshot[applied={T|T < 7315688078884847616}]
I20260806 22:50:59.888860 31692 mvcc.cc:205] Tried to move back new op lower bound from 7315688078884847616 to 7315688078786748416. Current Snapshot: MvccSnapshot[applied={T|T < 7315688078884847616}]
I20260806 22:50:59.890755 30993 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39424:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:50:59.890861 30993 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39424:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:50:59.891701 30993 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8b8567ff38284c8fb7318fb610cd1c6b: Sending DeleteTablet for 3 replicas of tablet acc3eeb66c5a43c0bcd31feabd986700
I20260806 22:50:59.892059 31549 tablet_service.cc:1514] Processing DeleteTablet for tablet acc3eeb66c5a43c0bcd31feabd986700 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:50:59 UTC) from {username='slave'} at 127.0.0.1:34530
I20260806 22:50:59.892097 31160 tablet_service.cc:1514] Processing DeleteTablet for tablet acc3eeb66c5a43c0bcd31feabd986700 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:50:59 UTC) from {username='slave'} at 127.0.0.1:49642
I20260806 22:50:59.892359 31365 tablet_service.cc:1514] Processing DeleteTablet for tablet acc3eeb66c5a43c0bcd31feabd986700 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:50:59 UTC) from {username='slave'} at 127.0.0.1:59486
I20260806 22:50:59.892385 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 31071
I20260806 22:50:59.900719 31710 tablet_replica.cc:330] T acc3eeb66c5a43c0bcd31feabd986700 P 6cf8ac809531456abd8a10183aec1a6c: stopping tablet replica
I20260806 22:50:59.900980 31710 raft_consensus.cc:2229] T acc3eeb66c5a43c0bcd31feabd986700 P 6cf8ac809531456abd8a10183aec1a6c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:50:59.901198 31710 raft_consensus.cc:2258] T acc3eeb66c5a43c0bcd31feabd986700 P 6cf8ac809531456abd8a10183aec1a6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:50:59.901827 31710 ts_tablet_manager.cc:1905] T acc3eeb66c5a43c0bcd31feabd986700 P 6cf8ac809531456abd8a10183aec1a6c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:50:59.904002 31710 ts_tablet_manager.cc:1918] T acc3eeb66c5a43c0bcd31feabd986700 P 6cf8ac809531456abd8a10183aec1a6c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:50:59.904126 31710 log.cc:1196] T acc3eeb66c5a43c0bcd31feabd986700 P 6cf8ac809531456abd8a10183aec1a6c: Deleting WAL directory at /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056405724609-6305-0/minicluster-data/ts-2/wal/wals/acc3eeb66c5a43c0bcd31feabd986700
I20260806 22:50:59.904505 31710 ts_tablet_manager.cc:1939] T acc3eeb66c5a43c0bcd31feabd986700 P 6cf8ac809531456abd8a10183aec1a6c: Deleting consensus metadata
I20260806 22:50:59.905614 30977 catalog_manager.cc:4841] TS 6cf8ac809531456abd8a10183aec1a6c (127.6.40.67:37267): tablet acc3eeb66c5a43c0bcd31feabd986700 (table test-table [id=07501f8d27cd46ce8e57877a250498ce]) successfully deleted
W20260806 22:50:59.907469 30977 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.40.65:35591 (error 108)
I20260806 22:50:59.908061 31661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:59.907898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.65:35591 (local address 127.0.0.1:49654)
0806 22:50:59.907968 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:59.908017 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.65:35591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260806 22:50:59.908143 31713 tablet_replica.cc:330] T acc3eeb66c5a43c0bcd31feabd986700 P 4e4d21ffa4864a8aa57fd9f79a2ea625: stopping tablet replica
I20260806 22:50:59.908141 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 31265
I20260806 22:50:59.908274 31713 raft_consensus.cc:2229] T acc3eeb66c5a43c0bcd31feabd986700 P 4e4d21ffa4864a8aa57fd9f79a2ea625 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:50:59.908420 31713 raft_consensus.cc:2258] T acc3eeb66c5a43c0bcd31feabd986700 P 4e4d21ffa4864a8aa57fd9f79a2ea625 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260806 22:50:59.914180 30977 catalog_manager.cc:4584] TS 74b0f79680204e4fb8f2793666866381 (127.6.40.65:35591): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet acc3eeb66c5a43c0bcd31feabd986700: Network error: Client connection negotiation failed: client connection to 127.6.40.65:35591: connect: Connection refused (error 111)
I20260806 22:50:59.919127 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 31459
I20260806 22:50:59.919225 31661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:59.919097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.66:37151 (local address 127.0.0.1:59492)
0806 22:50:59.919157 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:59.919189 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.66:37151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260806 22:50:59.930579 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 30949
2026-08-06T22:50:59Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (736 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:50:59 dist-test-slave-zh2d krb5kdc[31719](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:50:59 dist-test-slave-zh2d krb5kdc[31719](info): set up 2 sockets
Aug 06 22:50:59 dist-test-slave-zh2d krb5kdc[31719](info): commencing operation
krb5kdc: starting...
W20260806 22:51:01.149935 6305 mini_kdc.cc:121] Time spent starting KDC: real 1.194s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/krb5kdc/test-user.keytab.
Aug 06 22:51:01 dist-test-slave-zh2d krb5kdc[31719](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056661, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:51:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:51:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.40.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.40.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.40.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab.
Entry for principal kudu/127.6.40.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab.
I20260806 22:51:01.275882 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.126:34721
--webserver_interface=127.6.40.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab
--principal=kudu/127.6.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:42247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.40.126:34721
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:51:01.358217 31735 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:51:01.358423 31735 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:51:01.358445 31735 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:51:01.359623 31735 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:51:01.359668 31735 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:51:01.359683 31735 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:51:01.359695 31735 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:51:01.359748 31735 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:51:01.361305 31735 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:42247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.40.126:34721
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab
--principal=kudu/127.6.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.126:34721
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31735
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:51:01.361562 31735 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:51:01.361800 31735 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:51:01.366744 31742 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:51:01.366744 31743 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:51:01.366917 31735 server_base.cc:1034] running on GCE node
W20260806 22:51:01.367022 31747 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:51:01.367329 31735 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:51:01.367620 31735 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:51:01.368783 31735 hybrid_clock.cc:648] HybridClock initialized: now 1786056661368777 us; error 37 us; skew 500 ppm
Aug 06 22:51:01 dist-test-slave-zh2d krb5kdc[31719](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056661, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:51:01.371315 31735 init.cc:377] Logged in from keytab as kudu/127.6.40.126@KRBTEST.COM (short username kudu)
I20260806 22:51:01.371815 31735 webserver.cc:460] Webserver started at http://127.6.40.126:39049/ using document root <none> and password file <none>
I20260806 22:51:01.372059 31735 fs_manager.cc:359] Metadata directory not provided
I20260806 22:51:01.372123 31735 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:51:01.372275 31735 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:51:01.373395 31735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/minicluster-data/master-0/data/instance:
uuid: "df1fae6e497349f4a569754359241fa2"
format_stamp: "Formatted at 2026-08-06 22:51:01 on dist-test-slave-zh2d"
server_key: "ce47f91f2964b4640bc645db1a2875d2"
server_key_iv: "33ff1746eed38b1d4562e9ece945878f"
server_key_version: "encryptionkey@0"
I20260806 22:51:01.373755 31735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/minicluster-data/master-0/wal/instance:
uuid: "df1fae6e497349f4a569754359241fa2"
format_stamp: "Formatted at 2026-08-06 22:51:01 on dist-test-slave-zh2d"
server_key: "ce47f91f2964b4640bc645db1a2875d2"
server_key_iv: "33ff1746eed38b1d4562e9ece945878f"
server_key_version: "encryptionkey@0"
I20260806 22:51:01.375120 31735 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:51:01.375895 31754 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:51:01.376088 31735 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:51:01.376192 31735 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/minicluster-data/master-0/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/minicluster-data/master-0/wal
uuid: "df1fae6e497349f4a569754359241fa2"
format_stamp: "Formatted at 2026-08-06 22:51:01 on dist-test-slave-zh2d"
server_key: "ce47f91f2964b4640bc645db1a2875d2"
server_key_iv: "33ff1746eed38b1d4562e9ece945878f"
server_key_version: "encryptionkey@0"
I20260806 22:51:01.376269 31735 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:51:01.390367 31735 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:51:01.392203 31735 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:51:01.392356 31735 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:51:01.396556 31735 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.126:34721
I20260806 22:51:01.396656 31831 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.126:34721 every 8 connection(s)
I20260806 22:51:01.397034 31735 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
I20260806 22:51:01.397933 31832 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:51:01.400346 31832 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df1fae6e497349f4a569754359241fa2: Bootstrap starting.
I20260806 22:51:01.400915 31832 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P df1fae6e497349f4a569754359241fa2: Neither blocks nor log segments found. Creating new log.
I20260806 22:51:01.401201 31832 log.cc:824] T 00000000000000000000000000000000 P df1fae6e497349f4a569754359241fa2: Log is configured to *not* fsync() on all Append() calls
I20260806 22:51:01.401235 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 31735
I20260806 22:51:01.401363 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/minicluster-data/master-0/wal/instance
I20260806 22:51:01.401580 6305 external_mini_cluster.cc:1428] Setting key e46dd335034e9e4e21ec6ff130025ff8
I20260806 22:51:01.402424 31832 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df1fae6e497349f4a569754359241fa2: No bootstrap required, opened a new log
I20260806 22:51:01.405160 31832 raft_consensus.cc:348] T 00000000000000000000000000000000 P df1fae6e497349f4a569754359241fa2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df1fae6e497349f4a569754359241fa2" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 34721 } }
I20260806 22:51:01.405347 31832 raft_consensus.cc:374] T 00000000000000000000000000000000 P df1fae6e497349f4a569754359241fa2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:51:01.405405 31832 raft_consensus.cc:729] T 00000000000000000000000000000000 P df1fae6e497349f4a569754359241fa2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df1fae6e497349f4a569754359241fa2, State: Initialized, Role: FOLLOWER
I20260806 22:51:01.405543 31832 consensus_queue.cc:260] T 00000000000000000000000000000000 P df1fae6e497349f4a569754359241fa2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df1fae6e497349f4a569754359241fa2" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 34721 } }
I20260806 22:51:01.405611 31832 raft_consensus.cc:388] T 00000000000000000000000000000000 P df1fae6e497349f4a569754359241fa2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:51:01.405659 31832 raft_consensus.cc:482] T 00000000000000000000000000000000 P df1fae6e497349f4a569754359241fa2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:51:01.405714 31832 raft_consensus.cc:3037] T 00000000000000000000000000000000 P df1fae6e497349f4a569754359241fa2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:51:01.406334 31832 raft_consensus.cc:504] T 00000000000000000000000000000000 P df1fae6e497349f4a569754359241fa2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df1fae6e497349f4a569754359241fa2" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 34721 } }
I20260806 22:51:01.406448 31832 leader_election.cc:302] T 00000000000000000000000000000000 P df1fae6e497349f4a569754359241fa2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: df1fae6e497349f4a569754359241fa2; no voters:
I20260806 22:51:01.406613 31832 leader_election.cc:288] T 00000000000000000000000000000000 P df1fae6e497349f4a569754359241fa2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:51:01.406669 31840 raft_consensus.cc:2781] T 00000000000000000000000000000000 P df1fae6e497349f4a569754359241fa2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:51:01.406838 31840 raft_consensus.cc:686] T 00000000000000000000000000000000 P df1fae6e497349f4a569754359241fa2 [term 1 LEADER]: Becoming Leader. State: Replica: df1fae6e497349f4a569754359241fa2, State: Running, Role: LEADER
I20260806 22:51:01.406893 31832 sys_catalog.cc:564] T 00000000000000000000000000000000 P df1fae6e497349f4a569754359241fa2 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:51:01.406924 31840 consensus_queue.cc:237] T 00000000000000000000000000000000 P df1fae6e497349f4a569754359241fa2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df1fae6e497349f4a569754359241fa2" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 34721 } }
I20260806 22:51:01.407363 31841 sys_catalog.cc:455] T 00000000000000000000000000000000 P df1fae6e497349f4a569754359241fa2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "df1fae6e497349f4a569754359241fa2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df1fae6e497349f4a569754359241fa2" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 34721 } } }
I20260806 22:51:01.407456 31841 sys_catalog.cc:458] T 00000000000000000000000000000000 P df1fae6e497349f4a569754359241fa2 [sys.catalog]: This master's current role is: LEADER
I20260806 22:51:01.407351 31842 sys_catalog.cc:455] T 00000000000000000000000000000000 P df1fae6e497349f4a569754359241fa2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader df1fae6e497349f4a569754359241fa2. Latest consensus state: current_term: 1 leader_uuid: "df1fae6e497349f4a569754359241fa2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df1fae6e497349f4a569754359241fa2" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 34721 } } }
I20260806 22:51:01.407632 31842 sys_catalog.cc:458] T 00000000000000000000000000000000 P df1fae6e497349f4a569754359241fa2 [sys.catalog]: This master's current role is: LEADER
I20260806 22:51:01.408281 31847 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:51:01.408752 31847 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 06 22:51:01 dist-test-slave-zh2d krb5kdc[31719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056661, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
I20260806 22:51:01.410441 31847 catalog_manager.cc:1347] Generated new cluster ID: 7de512db486b41f4b57a51acd200c263
I20260806 22:51:01.410501 31847 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:51:01.414412 31839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:01.402100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57190 (local address 127.6.40.126:34721)
0806 22:51:01.402281 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:01.402290 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:01.402336 (+ 46us) server_negotiation.cc:407] Connection header received
0806 22:51:01.402927 (+ 591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:01.402936 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:01.403002 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:01.403163 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:01.403744 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:01.405205 (+ 1461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:01.406188 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:01.406831 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:01.408496 (+ 1665us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:01.410205 (+ 1709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:51:01.410209 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:51:01.410210 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:51:01.410222 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:51:01.412307 (+ 2085us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:01.412723 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:01.412726 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:01.412726 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:01.412829 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:01.413106 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:01.413107 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:01.413107 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:01.413406 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:51:01.413577 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:01.414048 (+ 471us) server_negotiation.cc:299] Negotiation successful
0806 22:51:01.414190 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":94,"threads_started":1}
I20260806 22:51:01.421813 31847 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:51:01.422508 31847 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:51:01.429646 31847 catalog_manager.cc:5868] T 00000000000000000000000000000000 P df1fae6e497349f4a569754359241fa2: Generated new TSK 0
I20260806 22:51:01.429836 31847 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.40.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.40.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.40.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.65.keytab.
Entry for principal kudu/127.6.40.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.65.keytab.
I20260806 22:51:01.463773 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.65:0
--local_ip_for_outbound_sockets=127.6.40.65
--webserver_interface=127.6.40.65
--webserver_port=0
--tserver_master_addrs=127.6.40.126:34721
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.65.keytab
--principal=kudu/127.6.40.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:42247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:51:01.548720 31874 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:51:01.548934 31874 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:51:01.548964 31874 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:51:01.550104 31874 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:51:01.550146 31874 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:51:01.550195 31874 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.65
I20260806 22:51:01.551625 31874 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:42247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.65.keytab
--principal=kudu/127.6.40.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.40.65
--webserver_port=0
--tserver_master_addrs=127.6.40.126:34721
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31874
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.65
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:51:01.551877 31874 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:51:01.552119 31874 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:51:01.557382 31879 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:51:01.557463 31874 server_base.cc:1034] running on GCE node
W20260806 22:51:01.557391 31891 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:51:01.557395 31880 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:51:01.557760 31874 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:51:01.557960 31874 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:51:01.559098 31874 hybrid_clock.cc:648] HybridClock initialized: now 1786056661559093 us; error 34 us; skew 500 ppm
Aug 06 22:51:01 dist-test-slave-zh2d krb5kdc[31719](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056661, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:51:01.561574 31874 init.cc:377] Logged in from keytab as kudu/127.6.40.65@KRBTEST.COM (short username kudu)
I20260806 22:51:01.561996 31874 webserver.cc:460] Webserver started at http://127.6.40.65:42945/ using document root <none> and password file <none>
I20260806 22:51:01.562227 31874 fs_manager.cc:359] Metadata directory not provided
I20260806 22:51:01.562291 31874 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:51:01.562436 31874 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:51:01.563480 31874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/minicluster-data/ts-0/data/instance:
uuid: "d4d1b17bf0ba422ea1c7e154c0b6181d"
format_stamp: "Formatted at 2026-08-06 22:51:01 on dist-test-slave-zh2d"
server_key: "b728bbab446e3c878c6ce8eb89cec211"
server_key_iv: "5ce094bd7adeff53c57b776428eba360"
server_key_version: "encryptionkey@0"
I20260806 22:51:01.563838 31874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/minicluster-data/ts-0/wal/instance:
uuid: "d4d1b17bf0ba422ea1c7e154c0b6181d"
format_stamp: "Formatted at 2026-08-06 22:51:01 on dist-test-slave-zh2d"
server_key: "b728bbab446e3c878c6ce8eb89cec211"
server_key_iv: "5ce094bd7adeff53c57b776428eba360"
server_key_version: "encryptionkey@0"
I20260806 22:51:01.565205 31874 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:51:01.565903 31900 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:51:01.566066 31874 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:51:01.566164 31874 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/minicluster-data/ts-0/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/minicluster-data/ts-0/wal
uuid: "d4d1b17bf0ba422ea1c7e154c0b6181d"
format_stamp: "Formatted at 2026-08-06 22:51:01 on dist-test-slave-zh2d"
server_key: "b728bbab446e3c878c6ce8eb89cec211"
server_key_iv: "5ce094bd7adeff53c57b776428eba360"
server_key_version: "encryptionkey@0"
I20260806 22:51:01.566234 31874 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:51:01.581409 31874 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:51:01.583173 31874 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:51:01.583325 31874 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:51:01.583549 31874 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:51:01.583843 31874 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:51:01.583894 31874 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:51:01.583935 31874 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:51:01.583963 31874 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:51:01.590230 31874 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.65:45969
I20260806 22:51:01.590292 32062 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.65:45969 every 8 connection(s)
I20260806 22:51:01.590628 31874 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
Aug 06 22:51:01 dist-test-slave-zh2d krb5kdc[31719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056661, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.65@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
I20260806 22:51:01.599342 31839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:01.591348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.65:56259 (local address 127.6.40.126:34721)
0806 22:51:01.591407 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:01.591409 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:01.591643 (+ 234us) server_negotiation.cc:407] Connection header received
0806 22:51:01.592122 (+ 479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:01.592123 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:01.592133 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:01.592190 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:01.593160 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:01.593665 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:01.594596 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:01.594782 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:01.594831 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:01.596958 (+ 2127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:51:01.596959 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:51:01.596959 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:51:01.596966 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:51:01.598176 (+ 1210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:01.598548 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:01.598549 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:01.598549 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:01.598598 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:01.598802 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:01.598802 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:01.598802 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:01.598900 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:51:01.598951 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:01.599169 (+ 218us) server_negotiation.cc:299] Negotiation successful
0806 22:51:01.599237 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 22:51:01.599481 32066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:01.591323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:34721 (local address 127.6.40.65:56259)
0806 22:51:01.591515 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0806 22:51:01.591558 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 22:51:01.592037 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:51:01.592242 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:51:01.592246 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:51:01.592812 (+ 566us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:51:01.593082 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:01.593087 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:01.593743 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:01.593744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:01.594524 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:01.594526 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:01.594850 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:01.594851 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:01.594948 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:01.595472 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:51:01.595480 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:51:01.596833 (+ 1353us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:51:01.598247 (+ 1414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:51:01.598254 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:51:01.598255 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:51:01.598487 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:51:01.598652 (+ 165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:51:01.598653 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:51:01.598653 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:51:01.598750 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:51:01.598951 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:51:01.598956 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:51:01.599066 (+ 110us) client_negotiation.cc:769] Sending connection context
0806 22:51:01.599146 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 22:51:01.599260 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":90,"threads_started":1}
I20260806 22:51:01.599481 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 31874
I20260806 22:51:01.599609 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/minicluster-data/ts-0/wal/instance
I20260806 22:51:01.599856 6305 external_mini_cluster.cc:1428] Setting key 9d0291816e4416ada646c2c1a3e4e83b
I20260806 22:51:01.599998 32064 heartbeater.cc:347] Connected to a master server at 127.6.40.126:34721
I20260806 22:51:01.600108 32064 heartbeater.cc:464] Registering TS with master...
I20260806 22:51:01.600297 32064 heartbeater.cc:511] Master 127.6.40.126:34721 requested a full tablet report, sending...
I20260806 22:51:01.600888 31780 ts_manager.cc:194] Registered new tserver with Master: d4d1b17bf0ba422ea1c7e154c0b6181d (127.6.40.65:45969)
I20260806 22:51:01.601497 31780 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.40.65@KRBTEST.COM'} at 127.6.40.65:56259
WARNING: no policy specified for kudu/127.6.40.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.40.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.40.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.66.keytab.
Entry for principal kudu/127.6.40.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.66.keytab.
I20260806 22:51:01.624874 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.66:0
--local_ip_for_outbound_sockets=127.6.40.66
--webserver_interface=127.6.40.66
--webserver_port=0
--tserver_master_addrs=127.6.40.126:34721
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.66.keytab
--principal=kudu/127.6.40.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:42247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:51:01.711196 32070 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:51:01.711395 32070 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:51:01.711417 32070 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:51:01.712545 32070 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:51:01.712587 32070 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:51:01.712613 32070 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.66
I20260806 22:51:01.714078 32070 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:42247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.66.keytab
--principal=kudu/127.6.40.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.40.66
--webserver_port=0
--tserver_master_addrs=127.6.40.126:34721
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32070
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.66
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:51:01.714311 32070 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:51:01.714547 32070 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:51:01.720140 32076 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:51:01.720146 32078 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:51:01.720162 32081 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:51:01.720430 32070 server_base.cc:1034] running on GCE node
I20260806 22:51:01.720656 32070 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:51:01.720984 32070 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:51:01.722138 32070 hybrid_clock.cc:648] HybridClock initialized: now 1786056661722115 us; error 36 us; skew 500 ppm
Aug 06 22:51:01 dist-test-slave-zh2d krb5kdc[31719](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056661, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:51:01.724923 32070 init.cc:377] Logged in from keytab as kudu/127.6.40.66@KRBTEST.COM (short username kudu)
I20260806 22:51:01.725414 32070 webserver.cc:460] Webserver started at http://127.6.40.66:46527/ using document root <none> and password file <none>
I20260806 22:51:01.725658 32070 fs_manager.cc:359] Metadata directory not provided
I20260806 22:51:01.725711 32070 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:51:01.725898 32070 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:51:01.727046 32070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/minicluster-data/ts-1/data/instance:
uuid: "f85c6fb4954645889d6bed4de0e98c86"
format_stamp: "Formatted at 2026-08-06 22:51:01 on dist-test-slave-zh2d"
server_key: "dbcaff2aa243f7947d4454e13df5681f"
server_key_iv: "41c15210719aaf7f15e28060528bdbb8"
server_key_version: "encryptionkey@0"
I20260806 22:51:01.727429 32070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/minicluster-data/ts-1/wal/instance:
uuid: "f85c6fb4954645889d6bed4de0e98c86"
format_stamp: "Formatted at 2026-08-06 22:51:01 on dist-test-slave-zh2d"
server_key: "dbcaff2aa243f7947d4454e13df5681f"
server_key_iv: "41c15210719aaf7f15e28060528bdbb8"
server_key_version: "encryptionkey@0"
I20260806 22:51:01.728965 32070 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:51:01.729851 32090 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:51:01.730046 32070 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:51:01.730139 32070 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/minicluster-data/ts-1/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/minicluster-data/ts-1/wal
uuid: "f85c6fb4954645889d6bed4de0e98c86"
format_stamp: "Formatted at 2026-08-06 22:51:01 on dist-test-slave-zh2d"
server_key: "dbcaff2aa243f7947d4454e13df5681f"
server_key_iv: "41c15210719aaf7f15e28060528bdbb8"
server_key_version: "encryptionkey@0"
I20260806 22:51:01.730217 32070 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:51:01.752981 32070 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:51:01.755158 32070 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:51:01.755313 32070 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:51:01.755570 32070 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:51:01.755867 32070 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:51:01.755899 32070 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:51:01.755939 32070 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:51:01.755965 32070 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:51:01.762125 32070 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.66:34031
I20260806 22:51:01.762205 32255 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.66:34031 every 8 connection(s)
I20260806 22:51:01.762529 32070 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
Aug 06 22:51:01 dist-test-slave-zh2d krb5kdc[31719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056661, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.66@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
I20260806 22:51:01.770308 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 32070
I20260806 22:51:01.770426 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/minicluster-data/ts-1/wal/instance
I20260806 22:51:01.770660 6305 external_mini_cluster.cc:1428] Setting key f1e0d5008869ddbe576e7ecb17df4235
I20260806 22:51:01.771896 31839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:01.763261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.66:54767 (local address 127.6.40.126:34721)
0806 22:51:01.763334 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:01.763337 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:01.763558 (+ 221us) server_negotiation.cc:407] Connection header received
0806 22:51:01.764078 (+ 520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:01.764079 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:01.764089 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:01.764133 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:01.765123 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:01.765618 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:01.766556 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:01.766739 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:01.766789 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:01.768897 (+ 2108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:51:01.768898 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:51:01.768899 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:51:01.768911 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:51:01.770324 (+ 1413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:01.770855 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:01.770857 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:01.770857 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:01.770933 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:01.771199 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:01.771200 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:01.771200 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:01.771359 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:51:01.771437 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:01.771764 (+ 327us) server_negotiation.cc:299] Negotiation successful
0806 22:51:01.771831 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 22:51:01.772265 32259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:01.763248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:34721 (local address 127.6.40.66:54767)
0806 22:51:01.763427 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0806 22:51:01.763465 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 22:51:01.763978 (+ 513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:51:01.764207 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:51:01.764210 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:51:01.764783 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:51:01.765046 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:01.765052 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:01.765693 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:01.765693 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:01.766477 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:01.766478 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:01.766809 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:01.766810 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:01.766907 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:01.767358 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:51:01.767366 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:51:01.768761 (+ 1395us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:51:01.770449 (+ 1688us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:51:01.770457 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:51:01.770460 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:51:01.770764 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:51:01.771016 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:51:01.771017 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:51:01.771017 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:51:01.771135 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:51:01.771440 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:51:01.771448 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:51:01.771614 (+ 166us) client_negotiation.cc:769] Sending connection context
0806 22:51:01.771729 (+ 115us) client_negotiation.cc:240] Negotiation successful
0806 22:51:01.771899 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":101,"threads_started":1}
I20260806 22:51:01.772774 32257 heartbeater.cc:347] Connected to a master server at 127.6.40.126:34721
I20260806 22:51:01.772902 32257 heartbeater.cc:464] Registering TS with master...
I20260806 22:51:01.773108 32257 heartbeater.cc:511] Master 127.6.40.126:34721 requested a full tablet report, sending...
I20260806 22:51:01.773469 31780 ts_manager.cc:194] Registered new tserver with Master: f85c6fb4954645889d6bed4de0e98c86 (127.6.40.66:34031)
I20260806 22:51:01.773847 31780 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.40.66@KRBTEST.COM'} at 127.6.40.66:54767
WARNING: no policy specified for kudu/127.6.40.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.40.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.40.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.67.keytab.
Entry for principal kudu/127.6.40.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.67.keytab.
I20260806 22:51:01.795912 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.67:0
--local_ip_for_outbound_sockets=127.6.40.67
--webserver_interface=127.6.40.67
--webserver_port=0
--tserver_master_addrs=127.6.40.126:34721
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.67.keytab
--principal=kudu/127.6.40.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:42247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:51:01.879148 32262 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:51:01.879375 32262 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:51:01.879415 32262 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:51:01.880661 32262 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:51:01.880718 32262 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:51:01.880764 32262 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.67
I20260806 22:51:01.882339 32262 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:42247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.67.keytab
--principal=kudu/127.6.40.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.40.67
--webserver_port=0
--tserver_master_addrs=127.6.40.126:34721
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32262
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.67
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:51:01.882607 32262 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:51:01.882833 32262 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:51:01.887986 32268 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:51:01.888028 32267 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:51:01.887987 32270 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:51:01.888001 32262 server_base.cc:1034] running on GCE node
I20260806 22:51:01.888371 32262 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:51:01.888635 32262 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:51:01.889875 32262 hybrid_clock.cc:648] HybridClock initialized: now 1786056661889856 us; error 28 us; skew 500 ppm
Aug 06 22:51:01 dist-test-slave-zh2d krb5kdc[31719](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056661, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:51:01.892416 32262 init.cc:377] Logged in from keytab as kudu/127.6.40.67@KRBTEST.COM (short username kudu)
I20260806 22:51:01.892901 32262 webserver.cc:460] Webserver started at http://127.6.40.67:41369/ using document root <none> and password file <none>
I20260806 22:51:01.893123 32262 fs_manager.cc:359] Metadata directory not provided
I20260806 22:51:01.893164 32262 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:51:01.893287 32262 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:51:01.894286 32262 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/minicluster-data/ts-2/data/instance:
uuid: "809be8babab04f6195ef7ca283a96f9f"
format_stamp: "Formatted at 2026-08-06 22:51:01 on dist-test-slave-zh2d"
server_key: "b060df51b17a3ec4f7670a8629af99bf"
server_key_iv: "a7dcdefb035c6fc5684c38014e963853"
server_key_version: "encryptionkey@0"
I20260806 22:51:01.894599 32262 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/minicluster-data/ts-2/wal/instance:
uuid: "809be8babab04f6195ef7ca283a96f9f"
format_stamp: "Formatted at 2026-08-06 22:51:01 on dist-test-slave-zh2d"
server_key: "b060df51b17a3ec4f7670a8629af99bf"
server_key_iv: "a7dcdefb035c6fc5684c38014e963853"
server_key_version: "encryptionkey@0"
I20260806 22:51:01.895848 32262 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:51:01.896513 32281 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:51:01.896669 32262 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:51:01.896755 32262 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/minicluster-data/ts-2/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/minicluster-data/ts-2/wal
uuid: "809be8babab04f6195ef7ca283a96f9f"
format_stamp: "Formatted at 2026-08-06 22:51:01 on dist-test-slave-zh2d"
server_key: "b060df51b17a3ec4f7670a8629af99bf"
server_key_iv: "a7dcdefb035c6fc5684c38014e963853"
server_key_version: "encryptionkey@0"
I20260806 22:51:01.896809 32262 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:51:01.916599 32262 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:51:01.919034 32262 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:51:01.919157 32262 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:51:01.919363 32262 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:51:01.919700 32262 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:51:01.919739 32262 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:51:01.919762 32262 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:51:01.919775 32262 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:51:01.925905 32262 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.67:36149
I20260806 22:51:01.925971 32443 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.67:36149 every 8 connection(s)
I20260806 22:51:01.926299 32262 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
I20260806 22:51:01.930958 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 32262
I20260806 22:51:01.931042 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/minicluster-data/ts-2/wal/instance
I20260806 22:51:01.931246 6305 external_mini_cluster.cc:1428] Setting key 9a4af57b9b5014eedd4d20ac0385b395
Aug 06 22:51:01 dist-test-slave-zh2d krb5kdc[31719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056661, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.67@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
I20260806 22:51:01.935925 31839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:01.926975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.67:35251 (local address 127.6.40.126:34721)
0806 22:51:01.927039 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:01.927041 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:01.927242 (+ 201us) server_negotiation.cc:407] Connection header received
0806 22:51:01.927757 (+ 515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:01.927758 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:01.927768 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:01.927812 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:01.928775 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:01.929392 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:01.930577 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:01.930783 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:01.930854 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:01.933277 (+ 2423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:51:01.933279 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:51:01.933279 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:51:01.933286 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:51:01.934586 (+ 1300us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:01.934986 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:01.934988 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:01.934988 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:01.935053 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:01.935301 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:01.935302 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:01.935303 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:01.935414 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:51:01.935489 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:01.935751 (+ 262us) server_negotiation.cc:299] Negotiation successful
0806 22:51:01.935833 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 22:51:01.936137 32449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:01.926938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:34721 (local address 127.6.40.67:35251)
0806 22:51:01.927119 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0806 22:51:01.927161 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 22:51:01.927656 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:51:01.927889 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:51:01.927892 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:51:01.928475 (+ 583us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:51:01.928669 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:01.928673 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:01.929509 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:01.929510 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:01.930486 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:01.930488 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:01.930862 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:01.930863 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:01.930978 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:01.931588 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:51:01.931598 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:51:01.933187 (+ 1589us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:51:01.934667 (+ 1480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:51:01.934676 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:51:01.934678 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:51:01.934898 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:51:01.935126 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:51:01.935127 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:51:01.935127 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:51:01.935237 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:51:01.935481 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:51:01.935487 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:51:01.935626 (+ 139us) client_negotiation.cc:769] Sending connection context
0806 22:51:01.935730 (+ 104us) client_negotiation.cc:240] Negotiation successful
0806 22:51:01.935888 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":104,"threads_started":1}
I20260806 22:51:01.936551 32444 heartbeater.cc:347] Connected to a master server at 127.6.40.126:34721
I20260806 22:51:01.936630 32444 heartbeater.cc:464] Registering TS with master...
I20260806 22:51:01.936808 32444 heartbeater.cc:511] Master 127.6.40.126:34721 requested a full tablet report, sending...
I20260806 22:51:01.937180 31780 ts_manager.cc:194] Registered new tserver with Master: 809be8babab04f6195ef7ca283a96f9f (127.6.40.67:36149)
I20260806 22:51:01.937553 31780 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.40.67@KRBTEST.COM'} at 127.6.40.67:35251
I20260806 22:51:01.943076 6305 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:51:01.950387 31839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:01.944506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57206 (local address 127.6.40.126:34721)
0806 22:51:01.944578 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:01.944580 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:01.944683 (+ 103us) server_negotiation.cc:407] Connection header received
0806 22:51:01.944722 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:01.944723 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:01.944731 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:01.944784 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:01.945351 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:01.945847 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:01.946547 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:01.946727 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:01.946787 (+ 60us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:01.948092 (+ 1305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:51:01.948093 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:51:01.948093 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:51:01.948099 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:51:01.949318 (+ 1219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:01.949672 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:01.949673 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:01.949673 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:01.949724 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:01.949909 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:01.949909 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:01.949909 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:01.950035 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:51:01.950115 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:01.950217 (+ 102us) server_negotiation.cc:299] Negotiation successful
0806 22:51:01.950268 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 22:51:01.952026 31780 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57206:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:51:01.953783 31780 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:51:01.960284 32462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:01.956179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.66:34031 (local address 127.0.0.1:57200)
0806 22:51:01.956593 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0806 22:51:01.956628 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 22:51:01.956695 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:51:01.957039 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:51:01.957043 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:51:01.957045 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:51:01.957112 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:01.957114 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:01.958023 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:01.958024 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:01.959149 (+ 1125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:01.959152 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:01.959988 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:01.959989 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:01.960071 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:01.960084 (+ 13us) client_negotiation.cc:769] Sending connection context
0806 22:51:01.960156 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 22:51:01.960210 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":124,"threads_started":1}
I20260806 22:51:01.960295 32464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:01.956402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57200 (local address 127.6.40.66:34031)
0806 22:51:01.956708 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:01.956712 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:01.956723 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:51:01.956785 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:01.956787 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:01.956801 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:01.956932 (+ 131us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:51:01.957199 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:01.957909 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:01.959248 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:01.959907 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:01.959988 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:01.960059 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:01.960170 (+ 111us) server_negotiation.cc:299] Negotiation successful
0806 22:51:01.960210 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266,"thread_start_us":93,"threads_started":1}
I20260806 22:51:01.960803 32461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:01.956184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.67:36149 (local address 127.0.0.1:58132)
0806 22:51:01.956787 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0806 22:51:01.956796 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:51:01.956859 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:51:01.957135 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:51:01.957136 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:51:01.957137 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:51:01.957172 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:01.957173 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:01.958468 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:01.958468 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:01.959590 (+ 1122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:01.959592 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:01.960540 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:01.960541 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:01.960625 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:01.960630 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:51:01.960689 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 22:51:01.960729 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":593,"mutex_wait_us":28,"thread_start_us":107,"threads_started":1}
I20260806 22:51:01.960893 32466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:01.956417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58132 (local address 127.6.40.67:36149)
0806 22:51:01.956640 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:01.956647 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:01.956920 (+ 273us) server_negotiation.cc:407] Connection header received
0806 22:51:01.956967 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:01.956969 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:01.956984 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:01.957049 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:51:01.957253 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:01.958354 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:01.959671 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:01.960443 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:01.960513 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:01.960569 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:01.960713 (+ 144us) server_negotiation.cc:299] Negotiation successful
0806 22:51:01.960760 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":123,"threads_started":1}
I20260806 22:51:01.961210 32158 tablet_service.cc:1467] Processing CreateTablet for tablet 753f4222fb894fbb9a74a1855f362e2f (DEFAULT_TABLE table=test-table [id=2f4de4122ea74097909411f272bfd3ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:51:01.961494 32463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:01.956172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.65:45969 (local address 127.0.0.1:60640)
0806 22:51:01.956353 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0806 22:51:01.956625 (+ 272us) client_negotiation.cc:174] Beginning negotiation
0806 22:51:01.956702 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:51:01.957081 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:51:01.957082 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:51:01.957084 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:51:01.957173 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:01.957175 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:01.958492 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:01.958492 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:01.959426 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:01.959429 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:01.961031 (+ 1602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:01.961032 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:01.961187 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:01.961192 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:51:01.961297 (+ 105us) client_negotiation.cc:240] Negotiation successful
0806 22:51:01.961336 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"mutex_wait_us":26,"thread_start_us":50,"threads_started":1}
I20260806 22:51:01.961573 32465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:01.956396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60640 (local address 127.6.40.65:45969)
0806 22:51:01.956609 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:01.956614 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:01.956695 (+ 81us) server_negotiation.cc:407] Connection header received
0806 22:51:01.956882 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:01.956885 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:01.956902 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:01.956990 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:51:01.957264 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:01.958370 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:01.959518 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:01.960219 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:01.960291 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:01.960823 (+ 532us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:01.961412 (+ 589us) server_negotiation.cc:299] Negotiation successful
0806 22:51:01.961464 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":100,"threads_started":1}
I20260806 22:51:01.961599 32158 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 753f4222fb894fbb9a74a1855f362e2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:51:01.962143 31970 tablet_service.cc:1467] Processing CreateTablet for tablet 753f4222fb894fbb9a74a1855f362e2f (DEFAULT_TABLE table=test-table [id=2f4de4122ea74097909411f272bfd3ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:51:01.962445 31970 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 753f4222fb894fbb9a74a1855f362e2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:51:01.964506 32469 tablet_bootstrap.cc:492] T 753f4222fb894fbb9a74a1855f362e2f P f85c6fb4954645889d6bed4de0e98c86: Bootstrap starting.
I20260806 22:51:01.964684 32347 tablet_service.cc:1467] Processing CreateTablet for tablet 753f4222fb894fbb9a74a1855f362e2f (DEFAULT_TABLE table=test-table [id=2f4de4122ea74097909411f272bfd3ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:51:01.964994 32347 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 753f4222fb894fbb9a74a1855f362e2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:51:01.965274 32469 tablet_bootstrap.cc:654] T 753f4222fb894fbb9a74a1855f362e2f P f85c6fb4954645889d6bed4de0e98c86: Neither blocks nor log segments found. Creating new log.
I20260806 22:51:01.965423 32470 tablet_bootstrap.cc:492] T 753f4222fb894fbb9a74a1855f362e2f P d4d1b17bf0ba422ea1c7e154c0b6181d: Bootstrap starting.
I20260806 22:51:01.965914 32469 log.cc:824] T 753f4222fb894fbb9a74a1855f362e2f P f85c6fb4954645889d6bed4de0e98c86: Log is configured to *not* fsync() on all Append() calls
I20260806 22:51:01.966168 32470 tablet_bootstrap.cc:654] T 753f4222fb894fbb9a74a1855f362e2f P d4d1b17bf0ba422ea1c7e154c0b6181d: Neither blocks nor log segments found. Creating new log.
I20260806 22:51:01.966699 32469 tablet_bootstrap.cc:492] T 753f4222fb894fbb9a74a1855f362e2f P f85c6fb4954645889d6bed4de0e98c86: No bootstrap required, opened a new log
I20260806 22:51:01.966781 32469 ts_tablet_manager.cc:1397] T 753f4222fb894fbb9a74a1855f362e2f P f85c6fb4954645889d6bed4de0e98c86: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:51:01.966813 32470 log.cc:824] T 753f4222fb894fbb9a74a1855f362e2f P d4d1b17bf0ba422ea1c7e154c0b6181d: Log is configured to *not* fsync() on all Append() calls
I20260806 22:51:01.967617 32470 tablet_bootstrap.cc:492] T 753f4222fb894fbb9a74a1855f362e2f P d4d1b17bf0ba422ea1c7e154c0b6181d: No bootstrap required, opened a new log
I20260806 22:51:01.967686 32470 ts_tablet_manager.cc:1397] T 753f4222fb894fbb9a74a1855f362e2f P d4d1b17bf0ba422ea1c7e154c0b6181d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:51:01.969010 32474 tablet_bootstrap.cc:492] T 753f4222fb894fbb9a74a1855f362e2f P 809be8babab04f6195ef7ca283a96f9f: Bootstrap starting.
I20260806 22:51:01.968989 32469 raft_consensus.cc:348] T 753f4222fb894fbb9a74a1855f362e2f P f85c6fb4954645889d6bed4de0e98c86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d1b17bf0ba422ea1c7e154c0b6181d" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 45969 } } peers { permanent_uuid: "f85c6fb4954645889d6bed4de0e98c86" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 34031 } } peers { permanent_uuid: "809be8babab04f6195ef7ca283a96f9f" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 36149 } }
I20260806 22:51:01.969182 32469 raft_consensus.cc:374] T 753f4222fb894fbb9a74a1855f362e2f P f85c6fb4954645889d6bed4de0e98c86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:51:01.969223 32469 raft_consensus.cc:729] T 753f4222fb894fbb9a74a1855f362e2f P f85c6fb4954645889d6bed4de0e98c86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f85c6fb4954645889d6bed4de0e98c86, State: Initialized, Role: FOLLOWER
I20260806 22:51:01.969372 32469 consensus_queue.cc:260] T 753f4222fb894fbb9a74a1855f362e2f P f85c6fb4954645889d6bed4de0e98c86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d1b17bf0ba422ea1c7e154c0b6181d" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 45969 } } peers { permanent_uuid: "f85c6fb4954645889d6bed4de0e98c86" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 34031 } } peers { permanent_uuid: "809be8babab04f6195ef7ca283a96f9f" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 36149 } }
I20260806 22:51:01.969659 32469 ts_tablet_manager.cc:1428] T 753f4222fb894fbb9a74a1855f362e2f P f85c6fb4954645889d6bed4de0e98c86: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:51:01.969592 32470 raft_consensus.cc:348] T 753f4222fb894fbb9a74a1855f362e2f P d4d1b17bf0ba422ea1c7e154c0b6181d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d1b17bf0ba422ea1c7e154c0b6181d" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 45969 } } peers { permanent_uuid: "f85c6fb4954645889d6bed4de0e98c86" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 34031 } } peers { permanent_uuid: "809be8babab04f6195ef7ca283a96f9f" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 36149 } }
I20260806 22:51:01.969771 32470 raft_consensus.cc:374] T 753f4222fb894fbb9a74a1855f362e2f P d4d1b17bf0ba422ea1c7e154c0b6181d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:51:01.969794 32474 tablet_bootstrap.cc:654] T 753f4222fb894fbb9a74a1855f362e2f P 809be8babab04f6195ef7ca283a96f9f: Neither blocks nor log segments found. Creating new log.
I20260806 22:51:01.969794 32257 heartbeater.cc:503] Master 127.6.40.126:34721 was elected leader, sending a full tablet report...
I20260806 22:51:01.969818 32470 raft_consensus.cc:729] T 753f4222fb894fbb9a74a1855f362e2f P d4d1b17bf0ba422ea1c7e154c0b6181d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4d1b17bf0ba422ea1c7e154c0b6181d, State: Initialized, Role: FOLLOWER
I20260806 22:51:01.969936 32470 consensus_queue.cc:260] T 753f4222fb894fbb9a74a1855f362e2f P d4d1b17bf0ba422ea1c7e154c0b6181d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d1b17bf0ba422ea1c7e154c0b6181d" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 45969 } } peers { permanent_uuid: "f85c6fb4954645889d6bed4de0e98c86" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 34031 } } peers { permanent_uuid: "809be8babab04f6195ef7ca283a96f9f" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 36149 } }
I20260806 22:51:01.970213 32470 ts_tablet_manager.cc:1428] T 753f4222fb894fbb9a74a1855f362e2f P d4d1b17bf0ba422ea1c7e154c0b6181d: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260806 22:51:01.970326 32064 heartbeater.cc:503] Master 127.6.40.126:34721 was elected leader, sending a full tablet report...
I20260806 22:51:01.970717 32474 log.cc:824] T 753f4222fb894fbb9a74a1855f362e2f P 809be8babab04f6195ef7ca283a96f9f: Log is configured to *not* fsync() on all Append() calls
I20260806 22:51:01.971411 32474 tablet_bootstrap.cc:492] T 753f4222fb894fbb9a74a1855f362e2f P 809be8babab04f6195ef7ca283a96f9f: No bootstrap required, opened a new log
I20260806 22:51:01.971467 32474 ts_tablet_manager.cc:1397] T 753f4222fb894fbb9a74a1855f362e2f P 809be8babab04f6195ef7ca283a96f9f: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 22:51:01.973029 32474 raft_consensus.cc:348] T 753f4222fb894fbb9a74a1855f362e2f P 809be8babab04f6195ef7ca283a96f9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d1b17bf0ba422ea1c7e154c0b6181d" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 45969 } } peers { permanent_uuid: "f85c6fb4954645889d6bed4de0e98c86" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 34031 } } peers { permanent_uuid: "809be8babab04f6195ef7ca283a96f9f" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 36149 } }
I20260806 22:51:01.973162 32474 raft_consensus.cc:374] T 753f4222fb894fbb9a74a1855f362e2f P 809be8babab04f6195ef7ca283a96f9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:51:01.973183 32474 raft_consensus.cc:729] T 753f4222fb894fbb9a74a1855f362e2f P 809be8babab04f6195ef7ca283a96f9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 809be8babab04f6195ef7ca283a96f9f, State: Initialized, Role: FOLLOWER
I20260806 22:51:01.973282 32474 consensus_queue.cc:260] T 753f4222fb894fbb9a74a1855f362e2f P 809be8babab04f6195ef7ca283a96f9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d1b17bf0ba422ea1c7e154c0b6181d" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 45969 } } peers { permanent_uuid: "f85c6fb4954645889d6bed4de0e98c86" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 34031 } } peers { permanent_uuid: "809be8babab04f6195ef7ca283a96f9f" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 36149 } }
I20260806 22:51:01.973493 32474 ts_tablet_manager.cc:1428] T 753f4222fb894fbb9a74a1855f362e2f P 809be8babab04f6195ef7ca283a96f9f: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:51:01.973559 32444 heartbeater.cc:503] Master 127.6.40.126:34721 was elected leader, sending a full tablet report...
W20260806 22:51:02.013118 32258 tablet.cc:2369] T 753f4222fb894fbb9a74a1855f362e2f P f85c6fb4954645889d6bed4de0e98c86: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:51:02.086967 32478 raft_consensus.cc:482] T 753f4222fb894fbb9a74a1855f362e2f P 809be8babab04f6195ef7ca283a96f9f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:51:02.087097 32478 raft_consensus.cc:504] T 753f4222fb894fbb9a74a1855f362e2f P 809be8babab04f6195ef7ca283a96f9f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d1b17bf0ba422ea1c7e154c0b6181d" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 45969 } } peers { permanent_uuid: "f85c6fb4954645889d6bed4de0e98c86" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 34031 } } peers { permanent_uuid: "809be8babab04f6195ef7ca283a96f9f" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 36149 } }
I20260806 22:51:02.087567 32478 leader_election.cc:288] T 753f4222fb894fbb9a74a1855f362e2f P 809be8babab04f6195ef7ca283a96f9f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4d1b17bf0ba422ea1c7e154c0b6181d (127.6.40.65:45969), f85c6fb4954645889d6bed4de0e98c86 (127.6.40.66:34031)
I20260806 22:51:02.090544 32465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:02.087854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.67:33169 (local address 127.6.40.65:45969)
0806 22:51:02.087913 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:02.087915 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:02.088133 (+ 218us) server_negotiation.cc:407] Connection header received
0806 22:51:02.088172 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:02.088173 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:02.088188 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:02.088255 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:51:02.088424 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:02.088892 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:02.089830 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:02.090191 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:02.090247 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:02.090275 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:02.090415 (+ 140us) server_negotiation.cc:299] Negotiation successful
0806 22:51:02.090466 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 22:51:02.090576 32464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:02.087895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.67:53879 (local address 127.6.40.66:34031)
0806 22:51:02.087950 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:02.087970 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:02.087978 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:51:02.088027 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:02.088028 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:02.088042 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:02.088114 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:51:02.088304 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:02.088827 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:02.089795 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:02.090204 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:02.090288 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:02.090298 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:02.090470 (+ 172us) server_negotiation.cc:299] Negotiation successful
0806 22:51:02.090513 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 22:51:02.090566 32480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:02.087780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.65:45969 (local address 127.6.40.67:33169)
0806 22:51:02.088052 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0806 22:51:02.088063 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:51:02.088124 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:51:02.088258 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:51:02.088272 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:51:02.088274 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:51:02.088355 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:02.088356 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:02.088987 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:02.088988 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:02.089715 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:02.089717 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:02.090270 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:02.090272 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:02.090345 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:02.090349 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:51:02.090391 (+ 42us) client_negotiation.cc:240] Negotiation successful
0806 22:51:02.090422 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"mutex_wait_us":58,"thread_start_us":140,"threads_started":1}
I20260806 22:51:02.090809 32449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:02.087780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.66:34031 (local address 127.6.40.67:53879)
0806 22:51:02.087844 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0806 22:51:02.087860 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 22:51:02.087927 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:51:02.088143 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:51:02.088144 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:51:02.088146 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:51:02.088228 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:02.088230 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:02.088949 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:02.088950 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:02.089705 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:02.089707 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:02.090305 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:02.090306 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:02.090392 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:02.090396 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:51:02.090453 (+ 57us) client_negotiation.cc:240] Negotiation successful
0806 22:51:02.090486 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260806 22:51:02.090965 32001 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "753f4222fb894fbb9a74a1855f362e2f" candidate_uuid: "809be8babab04f6195ef7ca283a96f9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d1b17bf0ba422ea1c7e154c0b6181d" is_pre_election: true
I20260806 22:51:02.091176 32001 raft_consensus.cc:2454] T 753f4222fb894fbb9a74a1855f362e2f P d4d1b17bf0ba422ea1c7e154c0b6181d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 809be8babab04f6195ef7ca283a96f9f in term 0.
I20260806 22:51:02.091200 32189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "753f4222fb894fbb9a74a1855f362e2f" candidate_uuid: "809be8babab04f6195ef7ca283a96f9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f85c6fb4954645889d6bed4de0e98c86" is_pre_election: true
I20260806 22:51:02.091348 32189 raft_consensus.cc:2454] T 753f4222fb894fbb9a74a1855f362e2f P f85c6fb4954645889d6bed4de0e98c86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 809be8babab04f6195ef7ca283a96f9f in term 0.
W20260806 22:51:02.091351 32065 tablet.cc:2369] T 753f4222fb894fbb9a74a1855f362e2f P d4d1b17bf0ba422ea1c7e154c0b6181d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:51:02.091465 32287 leader_election.cc:302] T 753f4222fb894fbb9a74a1855f362e2f P 809be8babab04f6195ef7ca283a96f9f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 809be8babab04f6195ef7ca283a96f9f, d4d1b17bf0ba422ea1c7e154c0b6181d; no voters:
I20260806 22:51:02.091598 32478 raft_consensus.cc:2781] T 753f4222fb894fbb9a74a1855f362e2f P 809be8babab04f6195ef7ca283a96f9f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:51:02.091670 32478 raft_consensus.cc:482] T 753f4222fb894fbb9a74a1855f362e2f P 809be8babab04f6195ef7ca283a96f9f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:51:02.091713 32478 raft_consensus.cc:3037] T 753f4222fb894fbb9a74a1855f362e2f P 809be8babab04f6195ef7ca283a96f9f [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:51:02.092433 32478 raft_consensus.cc:504] T 753f4222fb894fbb9a74a1855f362e2f P 809be8babab04f6195ef7ca283a96f9f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d1b17bf0ba422ea1c7e154c0b6181d" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 45969 } } peers { permanent_uuid: "f85c6fb4954645889d6bed4de0e98c86" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 34031 } } peers { permanent_uuid: "809be8babab04f6195ef7ca283a96f9f" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 36149 } }
I20260806 22:51:02.092581 32478 leader_election.cc:288] T 753f4222fb894fbb9a74a1855f362e2f P 809be8babab04f6195ef7ca283a96f9f [CANDIDATE]: Term 1 election: Requested vote from peers d4d1b17bf0ba422ea1c7e154c0b6181d (127.6.40.65:45969), f85c6fb4954645889d6bed4de0e98c86 (127.6.40.66:34031)
I20260806 22:51:02.092749 32189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "753f4222fb894fbb9a74a1855f362e2f" candidate_uuid: "809be8babab04f6195ef7ca283a96f9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f85c6fb4954645889d6bed4de0e98c86"
I20260806 22:51:02.092792 32001 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "753f4222fb894fbb9a74a1855f362e2f" candidate_uuid: "809be8babab04f6195ef7ca283a96f9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d1b17bf0ba422ea1c7e154c0b6181d"
I20260806 22:51:02.092832 32189 raft_consensus.cc:3037] T 753f4222fb894fbb9a74a1855f362e2f P f85c6fb4954645889d6bed4de0e98c86 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:51:02.092903 32001 raft_consensus.cc:3037] T 753f4222fb894fbb9a74a1855f362e2f P d4d1b17bf0ba422ea1c7e154c0b6181d [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:51:02.093685 32189 raft_consensus.cc:2454] T 753f4222fb894fbb9a74a1855f362e2f P f85c6fb4954645889d6bed4de0e98c86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 809be8babab04f6195ef7ca283a96f9f in term 1.
I20260806 22:51:02.093703 32001 raft_consensus.cc:2454] T 753f4222fb894fbb9a74a1855f362e2f P d4d1b17bf0ba422ea1c7e154c0b6181d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 809be8babab04f6195ef7ca283a96f9f in term 1.
I20260806 22:51:02.093866 32285 leader_election.cc:302] T 753f4222fb894fbb9a74a1855f362e2f P 809be8babab04f6195ef7ca283a96f9f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 809be8babab04f6195ef7ca283a96f9f, f85c6fb4954645889d6bed4de0e98c86; no voters:
I20260806 22:51:02.093981 32478 raft_consensus.cc:2781] T 753f4222fb894fbb9a74a1855f362e2f P 809be8babab04f6195ef7ca283a96f9f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:51:02.094142 32478 raft_consensus.cc:686] T 753f4222fb894fbb9a74a1855f362e2f P 809be8babab04f6195ef7ca283a96f9f [term 1 LEADER]: Becoming Leader. State: Replica: 809be8babab04f6195ef7ca283a96f9f, State: Running, Role: LEADER
I20260806 22:51:02.094244 32478 consensus_queue.cc:237] T 753f4222fb894fbb9a74a1855f362e2f P 809be8babab04f6195ef7ca283a96f9f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d1b17bf0ba422ea1c7e154c0b6181d" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 45969 } } peers { permanent_uuid: "f85c6fb4954645889d6bed4de0e98c86" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 34031 } } peers { permanent_uuid: "809be8babab04f6195ef7ca283a96f9f" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 36149 } }
I20260806 22:51:02.095085 31780 catalog_manager.cc:5495] T 753f4222fb894fbb9a74a1855f362e2f P 809be8babab04f6195ef7ca283a96f9f reported cstate change: term changed from 0 to 1, leader changed from <none> to 809be8babab04f6195ef7ca283a96f9f (127.6.40.67). New cstate: current_term: 1 leader_uuid: "809be8babab04f6195ef7ca283a96f9f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d1b17bf0ba422ea1c7e154c0b6181d" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 45969 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f85c6fb4954645889d6bed4de0e98c86" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 34031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "809be8babab04f6195ef7ca283a96f9f" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 36149 } health_report { overall_health: HEALTHY } } }
I20260806 22:51:02.104416 32466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:02.102265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58138 (local address 127.6.40.67:36149)
0806 22:51:02.102320 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:02.102322 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:02.102330 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:51:02.102380 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:02.102380 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:02.102399 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:02.102450 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:51:02.102556 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:02.103020 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:02.103714 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:02.103888 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:02.103937 (+ 49us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:02.104084 (+ 147us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:51:02.104167 (+ 83us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:51:02.104238 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:02.104282 (+ 44us) server_negotiation.cc:299] Negotiation successful
0806 22:51:02.104291 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 22:51:02.106292 32001 raft_consensus.cc:1261] T 753f4222fb894fbb9a74a1855f362e2f P d4d1b17bf0ba422ea1c7e154c0b6181d [term 1 FOLLOWER]: Refusing update from remote peer 809be8babab04f6195ef7ca283a96f9f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:51:02.106292 32189 raft_consensus.cc:1261] T 753f4222fb894fbb9a74a1855f362e2f P f85c6fb4954645889d6bed4de0e98c86 [term 1 FOLLOWER]: Refusing update from remote peer 809be8babab04f6195ef7ca283a96f9f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:51:02.106654 32481 consensus_queue.cc:1035] T 753f4222fb894fbb9a74a1855f362e2f P 809be8babab04f6195ef7ca283a96f9f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f85c6fb4954645889d6bed4de0e98c86" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 34031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:51:02.106767 32478 consensus_queue.cc:1035] T 753f4222fb894fbb9a74a1855f362e2f P 809be8babab04f6195ef7ca283a96f9f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4d1b17bf0ba422ea1c7e154c0b6181d" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 45969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:51:02.109233 32485 mvcc.cc:205] Tried to move back new op lower bound from 7315688087984885760 to 7315688087938715648. Current Snapshot: MvccSnapshot[applied={T|T < 7315688087984885760}]
I20260806 22:51:02.109813 32486 mvcc.cc:205] Tried to move back new op lower bound from 7315688087984885760 to 7315688087938715648. Current Snapshot: MvccSnapshot[applied={T|T < 7315688087984885760}]
I20260806 22:51:02.111932 31779 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57206:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:51:02.112020 31779 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57206:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:51:02.112787 31779 catalog_manager.cc:5782] T 00000000000000000000000000000000 P df1fae6e497349f4a569754359241fa2: Sending DeleteTablet for 3 replicas of tablet 753f4222fb894fbb9a74a1855f362e2f
I20260806 22:51:02.113176 32347 tablet_service.cc:1514] Processing DeleteTablet for tablet 753f4222fb894fbb9a74a1855f362e2f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:51:02 UTC) from {username='kudu', principal='kudu/127.6.40.126@KRBTEST.COM'} at 127.0.0.1:58132
I20260806 22:51:02.113174 32158 tablet_service.cc:1514] Processing DeleteTablet for tablet 753f4222fb894fbb9a74a1855f362e2f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:51:02 UTC) from {username='kudu', principal='kudu/127.6.40.126@KRBTEST.COM'} at 127.0.0.1:57200
I20260806 22:51:02.113391 32498 tablet_replica.cc:330] T 753f4222fb894fbb9a74a1855f362e2f P f85c6fb4954645889d6bed4de0e98c86: stopping tablet replica
I20260806 22:51:02.113512 32498 raft_consensus.cc:2229] T 753f4222fb894fbb9a74a1855f362e2f P f85c6fb4954645889d6bed4de0e98c86 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:51:02.113500 31970 tablet_service.cc:1514] Processing DeleteTablet for tablet 753f4222fb894fbb9a74a1855f362e2f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:51:02 UTC) from {username='kudu', principal='kudu/127.6.40.126@KRBTEST.COM'} at 127.0.0.1:60640
I20260806 22:51:02.113617 32498 raft_consensus.cc:2258] T 753f4222fb894fbb9a74a1855f362e2f P f85c6fb4954645889d6bed4de0e98c86 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:51:02.113667 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 31874
I20260806 22:51:02.114549 32498 ts_tablet_manager.cc:1905] T 753f4222fb894fbb9a74a1855f362e2f P f85c6fb4954645889d6bed4de0e98c86: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:51:02.116313 32498 ts_tablet_manager.cc:1918] T 753f4222fb894fbb9a74a1855f362e2f P f85c6fb4954645889d6bed4de0e98c86: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:51:02.116381 32498 log.cc:1196] T 753f4222fb894fbb9a74a1855f362e2f P f85c6fb4954645889d6bed4de0e98c86: Deleting WAL directory at /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/minicluster-data/ts-1/wal/wals/753f4222fb894fbb9a74a1855f362e2f
I20260806 22:51:02.116655 32498 ts_tablet_manager.cc:1939] T 753f4222fb894fbb9a74a1855f362e2f P f85c6fb4954645889d6bed4de0e98c86: Deleting consensus metadata
I20260806 22:51:02.117036 31759 catalog_manager.cc:4841] TS f85c6fb4954645889d6bed4de0e98c86 (127.6.40.66:34031): tablet 753f4222fb894fbb9a74a1855f362e2f (table test-table [id=2f4de4122ea74097909411f272bfd3ae]) successfully deleted
I20260806 22:51:02.122895 32500 tablet_replica.cc:330] T 753f4222fb894fbb9a74a1855f362e2f P 809be8babab04f6195ef7ca283a96f9f: stopping tablet replica
I20260806 22:51:02.123041 32500 raft_consensus.cc:2229] T 753f4222fb894fbb9a74a1855f362e2f P 809be8babab04f6195ef7ca283a96f9f [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:51:02.123196 32500 raft_consensus.cc:2258] T 753f4222fb894fbb9a74a1855f362e2f P 809be8babab04f6195ef7ca283a96f9f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:51:02.123718 32500 ts_tablet_manager.cc:1905] T 753f4222fb894fbb9a74a1855f362e2f P 809be8babab04f6195ef7ca283a96f9f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:51:02.125949 32500 ts_tablet_manager.cc:1918] T 753f4222fb894fbb9a74a1855f362e2f P 809be8babab04f6195ef7ca283a96f9f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:51:02.126031 32500 log.cc:1196] T 753f4222fb894fbb9a74a1855f362e2f P 809be8babab04f6195ef7ca283a96f9f: Deleting WAL directory at /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056405724609-6305-0/minicluster-data/ts-2/wal/wals/753f4222fb894fbb9a74a1855f362e2f
I20260806 22:51:02.126343 32500 ts_tablet_manager.cc:1939] T 753f4222fb894fbb9a74a1855f362e2f P 809be8babab04f6195ef7ca283a96f9f: Deleting consensus metadata
I20260806 22:51:02.127027 31756 catalog_manager.cc:4841] TS 809be8babab04f6195ef7ca283a96f9f (127.6.40.67:36149): tablet 753f4222fb894fbb9a74a1855f362e2f (table test-table [id=2f4de4122ea74097909411f272bfd3ae]) successfully deleted
W20260806 22:51:02.130785 31761 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.40.65:45969 (error 108)
I20260806 22:51:02.131042 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 32070
I20260806 22:51:02.134266 32463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:02.131321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.65:45969 (local address 127.0.0.1:60646)
0806 22:51:02.134150 (+ 2829us) negotiation.cc:107] Waiting for socket to connect
0806 22:51:02.134212 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.65:45969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2799}
W20260806 22:51:02.135322 31761 catalog_manager.cc:4584] TS d4d1b17bf0ba422ea1c7e154c0b6181d (127.6.40.65:45969): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 753f4222fb894fbb9a74a1855f362e2f: Network error: Client connection negotiation failed: client connection to 127.6.40.65:45969: connect: Connection refused (error 111)
I20260806 22:51:02.149828 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 32262
I20260806 22:51:02.162838 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 31735
2026-08-06T22:51:02Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2231 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-06T22:51:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:51:02Z Disabled control of system clock
I20260806 22:51:02.195541 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056405724609-6305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056405724609-6305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056405724609-6305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.126:38089
--webserver_interface=127.6.40.126
--webserver_port=0
--builtin_ntp_servers=127.6.40.84:45313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.40.126:38089
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 22:51:02.280711 32509 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:51:02.281008 32509 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:51:02.281036 32509 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:51:02.282258 32509 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:51:02.282317 32509 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:51:02.282333 32509 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:51:02.282348 32509 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:51:02.282362 32509 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:51:02.283656 32509 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:45313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056405724609-6305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056405724609-6305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.40.126:38089
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.126:38089
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32509
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056405724609-6305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:51:02.283862 32509 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:51:02.284139 32509 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:51:02.289288 32518 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:51:02.289278 32517 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:51:02.289260 32521 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:51:02.289535 32509 server_base.cc:1034] running on GCE node
I20260806 22:51:02.289811 32509 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:51:02.290100 32509 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:51:02.291275 32509 hybrid_clock.cc:648] HybridClock initialized: now 1786056662291248 us; error 36 us; skew 500 ppm
I20260806 22:51:02.297797 32509 webserver.cc:460] Webserver started at http://127.6.40.126:41679/ using document root <none> and password file <none>
I20260806 22:51:02.298058 32509 fs_manager.cc:359] Metadata directory not provided
I20260806 22:51:02.298110 32509 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:51:02.298243 32509 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:51:02.299319 32509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056405724609-6305-0/minicluster-data/master-0/data/instance:
uuid: "a34a18039d324175bdd1e18ac0d206b7"
format_stamp: "Formatted at 2026-08-06 22:51:02 on dist-test-slave-zh2d"
server_key: "42b4c3c2beefb2ff46ec4a5eeb512e5e"
server_key_iv: "68e0141c3d4836ae9ddca9e3a183f8c7"
server_key_version: "encryptionkey@0"
I20260806 22:51:02.299676 32509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056405724609-6305-0/minicluster-data/master-0/wal/instance:
uuid: "a34a18039d324175bdd1e18ac0d206b7"
format_stamp: "Formatted at 2026-08-06 22:51:02 on dist-test-slave-zh2d"
server_key: "42b4c3c2beefb2ff46ec4a5eeb512e5e"
server_key_iv: "68e0141c3d4836ae9ddca9e3a183f8c7"
server_key_version: "encryptionkey@0"
I20260806 22:51:02.301025 32509 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 22:51:02.301792 32529 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:51:02.301986 32509 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:51:02.302093 32509 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056405724609-6305-0/minicluster-data/master-0/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056405724609-6305-0/minicluster-data/master-0/wal
uuid: "a34a18039d324175bdd1e18ac0d206b7"
format_stamp: "Formatted at 2026-08-06 22:51:02 on dist-test-slave-zh2d"
server_key: "42b4c3c2beefb2ff46ec4a5eeb512e5e"
server_key_iv: "68e0141c3d4836ae9ddca9e3a183f8c7"
server_key_version: "encryptionkey@0"
I20260806 22:51:02.302167 32509 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056405724609-6305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056405724609-6305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056405724609-6305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:51:02.323251 32509 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:51:02.323599 32509 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:51:02.323755 32509 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:51:02.327895 32509 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.126:38089
I20260806 22:51:02.327935 32603 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.126:38089 every 8 connection(s)
I20260806 22:51:02.328316 32509 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
I20260806 22:51:02.329304 32604 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:51:02.330330 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 32509
I20260806 22:51:02.330448 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056405724609-6305-0/minicluster-data/master-0/wal/instance
I20260806 22:51:02.330698 6305 external_mini_cluster.cc:1428] Setting key 689ee9e894c598d56cc66074c17b0474
I20260806 22:51:02.331951 32604 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a34a18039d324175bdd1e18ac0d206b7: Bootstrap starting.
I20260806 22:51:02.332563 32604 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a34a18039d324175bdd1e18ac0d206b7: Neither blocks nor log segments found. Creating new log.
I20260806 22:51:02.332911 32604 log.cc:824] T 00000000000000000000000000000000 P a34a18039d324175bdd1e18ac0d206b7: Log is configured to *not* fsync() on all Append() calls
I20260806 22:51:02.333621 32604 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a34a18039d324175bdd1e18ac0d206b7: No bootstrap required, opened a new log
I20260806 22:51:02.335348 32604 raft_consensus.cc:348] T 00000000000000000000000000000000 P a34a18039d324175bdd1e18ac0d206b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a34a18039d324175bdd1e18ac0d206b7" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 38089 } }
I20260806 22:51:02.335497 32604 raft_consensus.cc:374] T 00000000000000000000000000000000 P a34a18039d324175bdd1e18ac0d206b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:51:02.335557 32604 raft_consensus.cc:729] T 00000000000000000000000000000000 P a34a18039d324175bdd1e18ac0d206b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a34a18039d324175bdd1e18ac0d206b7, State: Initialized, Role: FOLLOWER
I20260806 22:51:02.335685 32604 consensus_queue.cc:260] T 00000000000000000000000000000000 P a34a18039d324175bdd1e18ac0d206b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a34a18039d324175bdd1e18ac0d206b7" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 38089 } }
I20260806 22:51:02.335780 32604 raft_consensus.cc:388] T 00000000000000000000000000000000 P a34a18039d324175bdd1e18ac0d206b7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:51:02.335829 32604 raft_consensus.cc:482] T 00000000000000000000000000000000 P a34a18039d324175bdd1e18ac0d206b7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:51:02.335876 32604 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a34a18039d324175bdd1e18ac0d206b7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:51:02.336193 32610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:02.331241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38106 (local address 127.6.40.126:38089)
0806 22:51:02.331426 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:02.331433 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:02.331460 (+ 27us) server_negotiation.cc:407] Connection header received
0806 22:51:02.331858 (+ 398us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:02.331864 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:02.331922 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:02.332083 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:02.332291 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:02.333338 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:02.334181 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:02.334601 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:02.334732 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:02.335413 (+ 681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:51:02.335417 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:51:02.335418 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:51:02.335613 (+ 195us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:51:02.335655 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:51:02.335669 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:51:02.335741 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:02.335824 (+ 83us) server_negotiation.cc:299] Negotiation successful
0806 22:51:02.336018 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":83,"threads_started":1}
I20260806 22:51:02.336489 32604 raft_consensus.cc:504] T 00000000000000000000000000000000 P a34a18039d324175bdd1e18ac0d206b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a34a18039d324175bdd1e18ac0d206b7" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 38089 } }
I20260806 22:51:02.336608 32604 leader_election.cc:302] T 00000000000000000000000000000000 P a34a18039d324175bdd1e18ac0d206b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a34a18039d324175bdd1e18ac0d206b7; no voters:
I20260806 22:51:02.336759 32604 leader_election.cc:288] T 00000000000000000000000000000000 P a34a18039d324175bdd1e18ac0d206b7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:51:02.336813 32618 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a34a18039d324175bdd1e18ac0d206b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:51:02.337005 32618 raft_consensus.cc:686] T 00000000000000000000000000000000 P a34a18039d324175bdd1e18ac0d206b7 [term 1 LEADER]: Becoming Leader. State: Replica: a34a18039d324175bdd1e18ac0d206b7, State: Running, Role: LEADER
I20260806 22:51:02.337057 32604 sys_catalog.cc:564] T 00000000000000000000000000000000 P a34a18039d324175bdd1e18ac0d206b7 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:51:02.337110 32618 consensus_queue.cc:237] T 00000000000000000000000000000000 P a34a18039d324175bdd1e18ac0d206b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a34a18039d324175bdd1e18ac0d206b7" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 38089 } }
I20260806 22:51:02.337426 32618 sys_catalog.cc:455] T 00000000000000000000000000000000 P a34a18039d324175bdd1e18ac0d206b7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a34a18039d324175bdd1e18ac0d206b7. Latest consensus state: current_term: 1 leader_uuid: "a34a18039d324175bdd1e18ac0d206b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a34a18039d324175bdd1e18ac0d206b7" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 38089 } } }
I20260806 22:51:02.337523 32618 sys_catalog.cc:458] T 00000000000000000000000000000000 P a34a18039d324175bdd1e18ac0d206b7 [sys.catalog]: This master's current role is: LEADER
I20260806 22:51:02.337445 32621 sys_catalog.cc:455] T 00000000000000000000000000000000 P a34a18039d324175bdd1e18ac0d206b7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a34a18039d324175bdd1e18ac0d206b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a34a18039d324175bdd1e18ac0d206b7" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 38089 } } }
I20260806 22:51:02.337682 32621 sys_catalog.cc:458] T 00000000000000000000000000000000 P a34a18039d324175bdd1e18ac0d206b7 [sys.catalog]: This master's current role is: LEADER
I20260806 22:51:02.338295 32626 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:51:02.338797 32626 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:51:02.340251 32626 catalog_manager.cc:1347] Generated new cluster ID: 803304622e8f4e0da1e39327b4597533
I20260806 22:51:02.340296 32626 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:51:02.357357 32626 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:51:02.357903 32626 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:51:02.366146 32626 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a34a18039d324175bdd1e18ac0d206b7: Generated new TSK 0
I20260806 22:51:02.366329 32626 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:51:02.371143 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056405724609-6305-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056405724609-6305-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056405724609-6305-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.65:0
--local_ip_for_outbound_sockets=127.6.40.65
--webserver_interface=127.6.40.65
--webserver_port=0
--tserver_master_addrs=127.6.40.126:38089
--builtin_ntp_servers=127.6.40.84:45313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 22:51:02.456357 32647 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:51:02.456554 32647 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:51:02.456576 32647 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:51:02.457779 32647 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:51:02.457823 32647 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:51:02.457850 32647 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.65
I20260806 22:51:02.459213 32647 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:45313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056405724609-6305-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056405724609-6305-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.40.65
--webserver_port=0
--tserver_master_addrs=127.6.40.126:38089
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32647
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.65
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056405724609-6305-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:51:02.459470 32647 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:51:02.459777 32647 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:51:02.465041 32658 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:51:02.465037 32656 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:51:02.465102 32655 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:51:02.465139 32647 server_base.cc:1034] running on GCE node
I20260806 22:51:02.465443 32647 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:51:02.465709 32647 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:51:02.466878 32647 hybrid_clock.cc:648] HybridClock initialized: now 1786056662466866 us; error 36 us; skew 500 ppm
I20260806 22:51:02.468161 32647 webserver.cc:460] Webserver started at http://127.6.40.65:44203/ using document root <none> and password file <none>
I20260806 22:51:02.468436 32647 fs_manager.cc:359] Metadata directory not provided
I20260806 22:51:02.468523 32647 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:51:02.468667 32647 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:51:02.469787 32647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056405724609-6305-0/minicluster-data/ts-0/data/instance:
uuid: "c75a10f655c446e68bbf59db8d28a839"
format_stamp: "Formatted at 2026-08-06 22:51:02 on dist-test-slave-zh2d"
server_key: "e26729fd65b27ffb53824a6a126cc729"
server_key_iv: "dab3e101dc8206ca9e2540bce9edbceb"
server_key_version: "encryptionkey@0"
I20260806 22:51:02.470153 32647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056405724609-6305-0/minicluster-data/ts-0/wal/instance:
uuid: "c75a10f655c446e68bbf59db8d28a839"
format_stamp: "Formatted at 2026-08-06 22:51:02 on dist-test-slave-zh2d"
server_key: "e26729fd65b27ffb53824a6a126cc729"
server_key_iv: "dab3e101dc8206ca9e2540bce9edbceb"
server_key_version: "encryptionkey@0"
I20260806 22:51:02.471524 32647 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:51:02.472211 32666 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:51:02.472374 32647 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:51:02.472486 32647 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056405724609-6305-0/minicluster-data/ts-0/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056405724609-6305-0/minicluster-data/ts-0/wal
uuid: "c75a10f655c446e68bbf59db8d28a839"
format_stamp: "Formatted at 2026-08-06 22:51:02 on dist-test-slave-zh2d"
server_key: "e26729fd65b27ffb53824a6a126cc729"
server_key_iv: "dab3e101dc8206ca9e2540bce9edbceb"
server_key_version: "encryptionkey@0"
I20260806 22:51:02.472566 32647 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056405724609-6305-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056405724609-6305-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056405724609-6305-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:51:02.503223 32647 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:51:02.503609 32647 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:51:02.503772 32647 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:51:02.504033 32647 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:51:02.504395 32647 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:51:02.504447 32647 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:51:02.504499 32647 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:51:02.504522 32647 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:51:02.511076 32647 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.65:35535
I20260806 22:51:02.511122 376 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.65:35535 every 8 connection(s)
I20260806 22:51:02.511479 32647 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
I20260806 22:51:02.516650 32610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:02.512208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.65:37635 (local address 127.6.40.126:38089)
0806 22:51:02.512296 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:02.512299 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:02.512524 (+ 225us) server_negotiation.cc:407] Connection header received
0806 22:51:02.513126 (+ 602us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:02.513127 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:02.513137 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:02.513189 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:02.513881 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:02.514382 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:02.515378 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:02.515566 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:02.515615 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:02.516416 (+ 801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:51:02.516417 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:51:02.516417 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:51:02.516421 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:51:02.516434 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:51:02.516438 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:51:02.516482 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:02.516564 (+ 82us) server_negotiation.cc:299] Negotiation successful
0806 22:51:02.516584 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 22:51:02.516935 380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:02.512176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:38089 (local address 127.6.40.65:37635)
0806 22:51:02.512391 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0806 22:51:02.512430 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 22:51:02.513016 (+ 586us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:51:02.513254 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:51:02.513257 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:51:02.513266 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:51:02.513795 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:02.513800 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:02.514479 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:02.514481 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:02.515304 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:02.515306 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:02.515639 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:02.515640 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:02.515749 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:02.516287 (+ 538us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:51:02.516287 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:51:02.516316 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:51:02.516339 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:51:02.516492 (+ 153us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:51:02.516497 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:51:02.516499 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 22:51:02.516553 (+ 54us) client_negotiation.cc:240] Negotiation successful
0806 22:51:02.516635 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":109,"threads_started":1}
I20260806 22:51:02.517167 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 32647
I20260806 22:51:02.517288 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056405724609-6305-0/minicluster-data/ts-0/wal/instance
I20260806 22:51:02.517518 377 heartbeater.cc:347] Connected to a master server at 127.6.40.126:38089
I20260806 22:51:02.517544 6305 external_mini_cluster.cc:1428] Setting key c84d03d74f9855d179a860403846ed03
I20260806 22:51:02.517611 377 heartbeater.cc:464] Registering TS with master...
I20260806 22:51:02.517802 377 heartbeater.cc:511] Master 127.6.40.126:38089 requested a full tablet report, sending...
I20260806 22:51:02.518291 32551 ts_manager.cc:194] Registered new tserver with Master: c75a10f655c446e68bbf59db8d28a839 (127.6.40.65:35535)
I20260806 22:51:02.518546 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056405724609-6305-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056405724609-6305-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056405724609-6305-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.66:0
--local_ip_for_outbound_sockets=127.6.40.66
--webserver_interface=127.6.40.66
--webserver_port=0
--tserver_master_addrs=127.6.40.126:38089
--builtin_ntp_servers=127.6.40.84:45313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 22:51:02.518860 32551 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.40.65:37635
W20260806 22:51:02.604373 381 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:51:02.604585 381 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:51:02.604609 381 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:51:02.605849 381 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:51:02.605888 381 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:51:02.605950 381 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.66
I20260806 22:51:02.607311 381 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:45313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056405724609-6305-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056405724609-6305-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.40.66
--webserver_port=0
--tserver_master_addrs=127.6.40.126:38089
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.381
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.66
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056405724609-6305-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:51:02.607587 381 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:51:02.607893 381 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:51:02.613163 386 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:51:02.613180 393 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:51:02.613183 387 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:51:02.613467 381 server_base.cc:1034] running on GCE node
I20260806 22:51:02.613723 381 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:51:02.613972 381 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:51:02.615129 381 hybrid_clock.cc:648] HybridClock initialized: now 1786056662615129 us; error 41 us; skew 500 ppm
I20260806 22:51:02.616472 381 webserver.cc:460] Webserver started at http://127.6.40.66:43307/ using document root <none> and password file <none>
I20260806 22:51:02.616742 381 fs_manager.cc:359] Metadata directory not provided
I20260806 22:51:02.616812 381 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:51:02.616979 381 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:51:02.618067 381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056405724609-6305-0/minicluster-data/ts-1/data/instance:
uuid: "33dd8d7dd7694201948c484134dbbe71"
format_stamp: "Formatted at 2026-08-06 22:51:02 on dist-test-slave-zh2d"
server_key: "f28f4f47527824ff2958f5e61b618680"
server_key_iv: "24cc9e0059da05eb2a2273a3545d5f71"
server_key_version: "encryptionkey@0"
I20260806 22:51:02.618434 381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056405724609-6305-0/minicluster-data/ts-1/wal/instance:
uuid: "33dd8d7dd7694201948c484134dbbe71"
format_stamp: "Formatted at 2026-08-06 22:51:02 on dist-test-slave-zh2d"
server_key: "f28f4f47527824ff2958f5e61b618680"
server_key_iv: "24cc9e0059da05eb2a2273a3545d5f71"
server_key_version: "encryptionkey@0"
I20260806 22:51:02.619805 381 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:51:02.620569 401 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:51:02.620754 381 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:51:02.620885 381 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056405724609-6305-0/minicluster-data/ts-1/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056405724609-6305-0/minicluster-data/ts-1/wal
uuid: "33dd8d7dd7694201948c484134dbbe71"
format_stamp: "Formatted at 2026-08-06 22:51:02 on dist-test-slave-zh2d"
server_key: "f28f4f47527824ff2958f5e61b618680"
server_key_iv: "24cc9e0059da05eb2a2273a3545d5f71"
server_key_version: "encryptionkey@0"
I20260806 22:51:02.620968 381 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056405724609-6305-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056405724609-6305-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056405724609-6305-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:51:02.632102 381 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:51:02.632464 381 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:51:02.632624 381 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:51:02.632974 381 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:51:02.633356 381 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:51:02.633416 381 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:51:02.633453 381 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:51:02.633479 381 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:51:02.639921 381 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.66:36449
I20260806 22:51:02.639995 571 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.66:36449 every 8 connection(s)
I20260806 22:51:02.640352 381 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
I20260806 22:51:02.644286 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 381
I20260806 22:51:02.644378 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056405724609-6305-0/minicluster-data/ts-1/wal/instance
I20260806 22:51:02.644629 6305 external_mini_cluster.cc:1428] Setting key d8a5656d78520ed50372dfcc314bacaa
I20260806 22:51:02.645742 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056405724609-6305-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056405724609-6305-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056405724609-6305-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.67:0
--local_ip_for_outbound_sockets=127.6.40.67
--webserver_interface=127.6.40.67
--webserver_port=0
--tserver_master_addrs=127.6.40.126:38089
--builtin_ntp_servers=127.6.40.84:45313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 22:51:02.646059 32610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:02.641218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.66:50331 (local address 127.6.40.126:38089)
0806 22:51:02.641273 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:02.641275 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:02.641515 (+ 240us) server_negotiation.cc:407] Connection header received
0806 22:51:02.642084 (+ 569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:02.642086 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:02.642101 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:02.642161 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:02.642912 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:02.643432 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:02.644516 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:02.644751 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:02.644818 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:02.645774 (+ 956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:51:02.645775 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:51:02.645775 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:51:02.645780 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:51:02.645796 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:51:02.645800 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:51:02.645845 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:02.645943 (+ 98us) server_negotiation.cc:299] Negotiation successful
0806 22:51:02.645978 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 22:51:02.646292 575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:02.641144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:38089 (local address 127.6.40.66:50331)
0806 22:51:02.641378 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0806 22:51:02.641413 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 22:51:02.641971 (+ 558us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:51:02.642236 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:51:02.642240 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:51:02.642249 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:51:02.642812 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:02.642816 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:02.643534 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:02.643536 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:02.644369 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:02.644372 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:02.644827 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:02.644828 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:02.645030 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:02.645630 (+ 600us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:51:02.645631 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:51:02.645661 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:51:02.645685 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:51:02.645857 (+ 172us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:51:02.645861 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:51:02.645865 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:51:02.645945 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 22:51:02.646053 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":76,"threads_started":1}
I20260806 22:51:02.646790 572 heartbeater.cc:347] Connected to a master server at 127.6.40.126:38089
I20260806 22:51:02.646898 572 heartbeater.cc:464] Registering TS with master...
I20260806 22:51:02.647115 572 heartbeater.cc:511] Master 127.6.40.126:38089 requested a full tablet report, sending...
I20260806 22:51:02.647506 32551 ts_manager.cc:194] Registered new tserver with Master: 33dd8d7dd7694201948c484134dbbe71 (127.6.40.66:36449)
I20260806 22:51:02.647946 32551 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.40.66:50331
W20260806 22:51:02.731252 576 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:51:02.731474 576 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:51:02.731496 576 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:51:02.732604 576 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:51:02.732640 576 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:51:02.732666 576 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.67
I20260806 22:51:02.733991 576 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:45313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056405724609-6305-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056405724609-6305-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.40.67
--webserver_port=0
--tserver_master_addrs=127.6.40.126:38089
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.576
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.67
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056405724609-6305-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:51:02.734222 576 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:51:02.734491 576 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:51:02.740252 589 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:51:02.740310 576 server_base.cc:1034] running on GCE node
W20260806 22:51:02.740273 584 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:51:02.740273 586 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:51:02.740739 576 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:51:02.741060 576 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:51:02.742230 576 hybrid_clock.cc:648] HybridClock initialized: now 1786056662742213 us; error 33 us; skew 500 ppm
I20260806 22:51:02.743592 576 webserver.cc:460] Webserver started at http://127.6.40.67:42607/ using document root <none> and password file <none>
I20260806 22:51:02.743866 576 fs_manager.cc:359] Metadata directory not provided
I20260806 22:51:02.743953 576 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:51:02.744119 576 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:51:02.745337 576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056405724609-6305-0/minicluster-data/ts-2/data/instance:
uuid: "28134f440c8a413891363f444a1fa847"
format_stamp: "Formatted at 2026-08-06 22:51:02 on dist-test-slave-zh2d"
server_key: "2f67a4399b73db0256f6935739f8adf8"
server_key_iv: "82077744689e3cda298b4d03b197f68d"
server_key_version: "encryptionkey@0"
I20260806 22:51:02.745699 576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056405724609-6305-0/minicluster-data/ts-2/wal/instance:
uuid: "28134f440c8a413891363f444a1fa847"
format_stamp: "Formatted at 2026-08-06 22:51:02 on dist-test-slave-zh2d"
server_key: "2f67a4399b73db0256f6935739f8adf8"
server_key_iv: "82077744689e3cda298b4d03b197f68d"
server_key_version: "encryptionkey@0"
I20260806 22:51:02.747046 576 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:51:02.747759 599 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:51:02.747921 576 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:51:02.748013 576 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056405724609-6305-0/minicluster-data/ts-2/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056405724609-6305-0/minicluster-data/ts-2/wal
uuid: "28134f440c8a413891363f444a1fa847"
format_stamp: "Formatted at 2026-08-06 22:51:02 on dist-test-slave-zh2d"
server_key: "2f67a4399b73db0256f6935739f8adf8"
server_key_iv: "82077744689e3cda298b4d03b197f68d"
server_key_version: "encryptionkey@0"
I20260806 22:51:02.748082 576 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056405724609-6305-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056405724609-6305-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056405724609-6305-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:51:02.766304 576 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:51:02.766667 576 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:51:02.766824 576 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:51:02.767079 576 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:51:02.767438 576 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:51:02.767478 576 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:51:02.767519 576 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:51:02.767546 576 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:51:02.773787 576 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.67:38247
I20260806 22:51:02.773834 750 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.67:38247 every 8 connection(s)
I20260806 22:51:02.774185 576 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
I20260806 22:51:02.779423 32610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:02.774967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.67:38691 (local address 127.6.40.126:38089)
0806 22:51:02.775044 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:02.775046 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:02.775281 (+ 235us) server_negotiation.cc:407] Connection header received
0806 22:51:02.775835 (+ 554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:02.775836 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:02.775846 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:02.775913 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:02.776555 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:02.777105 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:02.778114 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:02.778346 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:02.778423 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:02.779192 (+ 769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:51:02.779193 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:51:02.779193 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:51:02.779198 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:51:02.779213 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:51:02.779217 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:51:02.779261 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:02.779340 (+ 79us) server_negotiation.cc:299] Negotiation successful
0806 22:51:02.779363 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 22:51:02.779623 754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:02.774959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:38089 (local address 127.6.40.67:38691)
0806 22:51:02.775148 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0806 22:51:02.775191 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 22:51:02.775755 (+ 564us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:51:02.775968 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:51:02.775971 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:51:02.775978 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:51:02.776482 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:02.776486 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:02.777189 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:02.777190 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:02.777999 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:02.778001 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:02.778452 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:02.778452 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:02.778567 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:02.779061 (+ 494us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:51:02.779062 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:51:02.779090 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:51:02.779112 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:51:02.779270 (+ 158us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:51:02.779274 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:51:02.779277 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:51:02.779329 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 22:51:02.779411 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":89,"threads_started":1}
I20260806 22:51:02.780128 752 heartbeater.cc:347] Connected to a master server at 127.6.40.126:38089
I20260806 22:51:02.780231 752 heartbeater.cc:464] Registering TS with master...
I20260806 22:51:02.780416 752 heartbeater.cc:511] Master 127.6.40.126:38089 requested a full tablet report, sending...
I20260806 22:51:02.780776 32551 ts_manager.cc:194] Registered new tserver with Master: 28134f440c8a413891363f444a1fa847 (127.6.40.67:38247)
I20260806 22:51:02.781157 32551 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.40.67:38691
I20260806 22:51:02.782058 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 576
I20260806 22:51:02.782168 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056405724609-6305-0/minicluster-data/ts-2/wal/instance
I20260806 22:51:02.782421 6305 external_mini_cluster.cc:1428] Setting key 054d8e13b159f1287cdcb97d13d287d2
I20260806 22:51:02.783298 6305 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:51:02.787258 32610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:02.784707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38110 (local address 127.6.40.126:38089)
0806 22:51:02.784763 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:02.784765 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:02.784880 (+ 115us) server_negotiation.cc:407] Connection header received
0806 22:51:02.784975 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:02.784976 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:02.784984 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:02.785031 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:02.785153 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:02.785588 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:02.786273 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:02.786441 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:02.786490 (+ 49us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:02.787063 (+ 573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:51:02.787064 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:51:02.787064 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:51:02.787068 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:51:02.787079 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:51:02.787083 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:51:02.787153 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:02.787197 (+ 44us) server_negotiation.cc:299] Negotiation successful
0806 22:51:02.787206 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 22:51:02.788861 32551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38110:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:51:02.790453 32551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:51:02.796561 762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:02.792562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.67:38247 (local address 127.0.0.1:59448)
0806 22:51:02.792697 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0806 22:51:02.792740 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 22:51:02.792806 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:51:02.793337 (+ 531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:51:02.793338 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:51:02.793341 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:51:02.793390 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:02.793392 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:02.794290 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:02.794291 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:02.795489 (+ 1198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:02.795491 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:02.796276 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:02.796277 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:02.796354 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:02.796366 (+ 12us) client_negotiation.cc:769] Sending connection context
0806 22:51:02.796435 (+ 69us) client_negotiation.cc:240] Negotiation successful
0806 22:51:02.796485 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"mutex_wait_us":1,"thread_start_us":81,"threads_started":1}
I20260806 22:51:02.796572 766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:02.792671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59448 (local address 127.6.40.67:38247)
0806 22:51:02.793111 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:02.793116 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:02.793127 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:51:02.793168 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:02.793170 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:02.793185 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:02.793267 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:51:02.793474 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:02.794200 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:02.795563 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:02.796190 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:02.796274 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:02.796364 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:02.796446 (+ 82us) server_negotiation.cc:299] Negotiation successful
0806 22:51:02.796482 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":396,"thread_start_us":98,"threads_started":1}
I20260806 22:51:02.796999 765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:02.792562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.65:35535 (local address 127.0.0.1:36868)
0806 22:51:02.792754 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0806 22:51:02.792764 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:51:02.792813 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:51:02.793315 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:51:02.793318 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:51:02.793321 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:51:02.793386 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:02.793388 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:02.794696 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:02.794697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:02.795717 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:02.795719 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:02.796752 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:02.796753 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:02.796816 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:02.796820 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:51:02.796878 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 22:51:02.796917 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"mutex_wait_us":3,"spinlock_wait_cycles":768,"thread_start_us":97,"threads_started":1}
I20260806 22:51:02.797115 767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:02.792786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.66:36449 (local address 127.0.0.1:41036)
0806 22:51:02.792938 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 22:51:02.792947 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:51:02.792984 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:51:02.793567 (+ 583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:51:02.793568 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:51:02.793569 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:51:02.793602 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:02.793603 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:02.794837 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:02.794838 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:02.795783 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:02.795785 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:02.796869 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:02.796870 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:02.796923 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:02.796926 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:51:02.796967 (+ 41us) client_negotiation.cc:240] Negotiation successful
0806 22:51:02.796999 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"mutex_wait_us":1,"thread_start_us":96,"threads_started":1}
I20260806 22:51:02.797317 769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:02.793074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41036 (local address 127.6.40.66:36449)
0806 22:51:02.793229 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:02.793234 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:02.793245 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:51:02.793434 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:02.793436 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:02.793453 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:02.793674 (+ 221us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:51:02.793736 (+ 62us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:02.794753 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:02.795865 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:02.796545 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:02.797046 (+ 501us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:02.797156 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:02.797185 (+ 29us) server_negotiation.cc:299] Negotiation successful
0806 22:51:02.797231 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":77,"threads_started":1}
I20260806 22:51:02.797475 768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:02.792906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36868 (local address 127.6.40.65:35535)
0806 22:51:02.793123 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:02.793128 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:02.793138 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:51:02.793174 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:02.793176 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:02.793191 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:02.793297 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:51:02.793478 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:02.794608 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:02.795800 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:02.796544 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:02.797231 (+ 687us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:02.797317 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:02.797342 (+ 25us) server_negotiation.cc:299] Negotiation successful
0806 22:51:02.797386 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":120,"threads_started":1}
I20260806 22:51:02.797395 662 tablet_service.cc:1467] Processing CreateTablet for tablet ce7f4bfca3ad45108e99539e4e27eaca (DEFAULT_TABLE table=test-table [id=72b8c6d561874ed7ab49204b830ff9ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:51:02.797804 662 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce7f4bfca3ad45108e99539e4e27eaca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:51:02.797956 475 tablet_service.cc:1467] Processing CreateTablet for tablet ce7f4bfca3ad45108e99539e4e27eaca (DEFAULT_TABLE table=test-table [id=72b8c6d561874ed7ab49204b830ff9ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:51:02.798033 32737 tablet_service.cc:1467] Processing CreateTablet for tablet ce7f4bfca3ad45108e99539e4e27eaca (DEFAULT_TABLE table=test-table [id=72b8c6d561874ed7ab49204b830ff9ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:51:02.798285 32737 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce7f4bfca3ad45108e99539e4e27eaca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:51:02.798287 475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce7f4bfca3ad45108e99539e4e27eaca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:51:02.800800 770 tablet_bootstrap.cc:492] T ce7f4bfca3ad45108e99539e4e27eaca P c75a10f655c446e68bbf59db8d28a839: Bootstrap starting.
I20260806 22:51:02.801733 771 tablet_bootstrap.cc:492] T ce7f4bfca3ad45108e99539e4e27eaca P 33dd8d7dd7694201948c484134dbbe71: Bootstrap starting.
I20260806 22:51:02.801944 770 tablet_bootstrap.cc:654] T ce7f4bfca3ad45108e99539e4e27eaca P c75a10f655c446e68bbf59db8d28a839: Neither blocks nor log segments found. Creating new log.
I20260806 22:51:02.802341 770 log.cc:824] T ce7f4bfca3ad45108e99539e4e27eaca P c75a10f655c446e68bbf59db8d28a839: Log is configured to *not* fsync() on all Append() calls
I20260806 22:51:02.802416 771 tablet_bootstrap.cc:654] T ce7f4bfca3ad45108e99539e4e27eaca P 33dd8d7dd7694201948c484134dbbe71: Neither blocks nor log segments found. Creating new log.
I20260806 22:51:02.802771 771 log.cc:824] T ce7f4bfca3ad45108e99539e4e27eaca P 33dd8d7dd7694201948c484134dbbe71: Log is configured to *not* fsync() on all Append() calls
I20260806 22:51:02.803123 770 tablet_bootstrap.cc:492] T ce7f4bfca3ad45108e99539e4e27eaca P c75a10f655c446e68bbf59db8d28a839: No bootstrap required, opened a new log
I20260806 22:51:02.803215 770 ts_tablet_manager.cc:1397] T ce7f4bfca3ad45108e99539e4e27eaca P c75a10f655c446e68bbf59db8d28a839: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 22:51:02.803483 771 tablet_bootstrap.cc:492] T ce7f4bfca3ad45108e99539e4e27eaca P 33dd8d7dd7694201948c484134dbbe71: No bootstrap required, opened a new log
I20260806 22:51:02.803552 771 ts_tablet_manager.cc:1397] T ce7f4bfca3ad45108e99539e4e27eaca P 33dd8d7dd7694201948c484134dbbe71: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:51:02.806119 770 raft_consensus.cc:348] T ce7f4bfca3ad45108e99539e4e27eaca P c75a10f655c446e68bbf59db8d28a839 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28134f440c8a413891363f444a1fa847" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 38247 } } peers { permanent_uuid: "c75a10f655c446e68bbf59db8d28a839" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 35535 } } peers { permanent_uuid: "33dd8d7dd7694201948c484134dbbe71" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 36449 } }
I20260806 22:51:02.806316 770 raft_consensus.cc:374] T ce7f4bfca3ad45108e99539e4e27eaca P c75a10f655c446e68bbf59db8d28a839 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:51:02.806356 770 raft_consensus.cc:729] T ce7f4bfca3ad45108e99539e4e27eaca P c75a10f655c446e68bbf59db8d28a839 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c75a10f655c446e68bbf59db8d28a839, State: Initialized, Role: FOLLOWER
I20260806 22:51:02.806479 770 consensus_queue.cc:260] T ce7f4bfca3ad45108e99539e4e27eaca P c75a10f655c446e68bbf59db8d28a839 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28134f440c8a413891363f444a1fa847" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 38247 } } peers { permanent_uuid: "c75a10f655c446e68bbf59db8d28a839" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 35535 } } peers { permanent_uuid: "33dd8d7dd7694201948c484134dbbe71" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 36449 } }
I20260806 22:51:02.806529 771 raft_consensus.cc:348] T ce7f4bfca3ad45108e99539e4e27eaca P 33dd8d7dd7694201948c484134dbbe71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28134f440c8a413891363f444a1fa847" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 38247 } } peers { permanent_uuid: "c75a10f655c446e68bbf59db8d28a839" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 35535 } } peers { permanent_uuid: "33dd8d7dd7694201948c484134dbbe71" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 36449 } }
I20260806 22:51:02.806689 771 raft_consensus.cc:374] T ce7f4bfca3ad45108e99539e4e27eaca P 33dd8d7dd7694201948c484134dbbe71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:51:02.806733 771 raft_consensus.cc:729] T ce7f4bfca3ad45108e99539e4e27eaca P 33dd8d7dd7694201948c484134dbbe71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33dd8d7dd7694201948c484134dbbe71, State: Initialized, Role: FOLLOWER
I20260806 22:51:02.806783 770 ts_tablet_manager.cc:1428] T ce7f4bfca3ad45108e99539e4e27eaca P c75a10f655c446e68bbf59db8d28a839: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260806 22:51:02.806833 377 heartbeater.cc:503] Master 127.6.40.126:38089 was elected leader, sending a full tablet report...
I20260806 22:51:02.802724 775 tablet_bootstrap.cc:492] T ce7f4bfca3ad45108e99539e4e27eaca P 28134f440c8a413891363f444a1fa847: Bootstrap starting.
I20260806 22:51:02.806852 771 consensus_queue.cc:260] T ce7f4bfca3ad45108e99539e4e27eaca P 33dd8d7dd7694201948c484134dbbe71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28134f440c8a413891363f444a1fa847" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 38247 } } peers { permanent_uuid: "c75a10f655c446e68bbf59db8d28a839" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 35535 } } peers { permanent_uuid: "33dd8d7dd7694201948c484134dbbe71" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 36449 } }
I20260806 22:51:02.807116 771 ts_tablet_manager.cc:1428] T ce7f4bfca3ad45108e99539e4e27eaca P 33dd8d7dd7694201948c484134dbbe71: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260806 22:51:02.807176 572 heartbeater.cc:503] Master 127.6.40.126:38089 was elected leader, sending a full tablet report...
I20260806 22:51:02.807632 775 tablet_bootstrap.cc:654] T ce7f4bfca3ad45108e99539e4e27eaca P 28134f440c8a413891363f444a1fa847: Neither blocks nor log segments found. Creating new log.
I20260806 22:51:02.808015 775 log.cc:824] T ce7f4bfca3ad45108e99539e4e27eaca P 28134f440c8a413891363f444a1fa847: Log is configured to *not* fsync() on all Append() calls
I20260806 22:51:02.808789 775 tablet_bootstrap.cc:492] T ce7f4bfca3ad45108e99539e4e27eaca P 28134f440c8a413891363f444a1fa847: No bootstrap required, opened a new log
I20260806 22:51:02.808898 775 ts_tablet_manager.cc:1397] T ce7f4bfca3ad45108e99539e4e27eaca P 28134f440c8a413891363f444a1fa847: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260806 22:51:02.810971 775 raft_consensus.cc:348] T ce7f4bfca3ad45108e99539e4e27eaca P 28134f440c8a413891363f444a1fa847 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28134f440c8a413891363f444a1fa847" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 38247 } } peers { permanent_uuid: "c75a10f655c446e68bbf59db8d28a839" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 35535 } } peers { permanent_uuid: "33dd8d7dd7694201948c484134dbbe71" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 36449 } }
I20260806 22:51:02.811121 775 raft_consensus.cc:374] T ce7f4bfca3ad45108e99539e4e27eaca P 28134f440c8a413891363f444a1fa847 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:51:02.811169 775 raft_consensus.cc:729] T ce7f4bfca3ad45108e99539e4e27eaca P 28134f440c8a413891363f444a1fa847 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28134f440c8a413891363f444a1fa847, State: Initialized, Role: FOLLOWER
I20260806 22:51:02.811285 775 consensus_queue.cc:260] T ce7f4bfca3ad45108e99539e4e27eaca P 28134f440c8a413891363f444a1fa847 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28134f440c8a413891363f444a1fa847" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 38247 } } peers { permanent_uuid: "c75a10f655c446e68bbf59db8d28a839" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 35535 } } peers { permanent_uuid: "33dd8d7dd7694201948c484134dbbe71" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 36449 } }
I20260806 22:51:02.811566 775 ts_tablet_manager.cc:1428] T ce7f4bfca3ad45108e99539e4e27eaca P 28134f440c8a413891363f444a1fa847: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:51:02.811687 752 heartbeater.cc:503] Master 127.6.40.126:38089 was elected leader, sending a full tablet report...
I20260806 22:51:02.812105 784 raft_consensus.cc:482] T ce7f4bfca3ad45108e99539e4e27eaca P 33dd8d7dd7694201948c484134dbbe71 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:51:02.812207 784 raft_consensus.cc:504] T ce7f4bfca3ad45108e99539e4e27eaca P 33dd8d7dd7694201948c484134dbbe71 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28134f440c8a413891363f444a1fa847" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 38247 } } peers { permanent_uuid: "c75a10f655c446e68bbf59db8d28a839" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 35535 } } peers { permanent_uuid: "33dd8d7dd7694201948c484134dbbe71" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 36449 } }
I20260806 22:51:02.812556 784 leader_election.cc:288] T ce7f4bfca3ad45108e99539e4e27eaca P 33dd8d7dd7694201948c484134dbbe71 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 28134f440c8a413891363f444a1fa847 (127.6.40.67:38247), c75a10f655c446e68bbf59db8d28a839 (127.6.40.65:35535)
I20260806 22:51:02.816496 766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:02.812870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.66:34163 (local address 127.6.40.67:38247)
0806 22:51:02.812914 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:02.812916 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:02.812928 (+ 12us) server_negotiation.cc:407] Connection header received
0806 22:51:02.812982 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:02.812983 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:02.812993 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:02.813064 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:51:02.813260 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:02.814002 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:02.815469 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:02.816029 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:02.816103 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:02.816113 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:02.816395 (+ 282us) server_negotiation.cc:299] Negotiation successful
0806 22:51:02.816432 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 22:51:02.816517 768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:02.812760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.66:39451 (local address 127.6.40.65:35535)
0806 22:51:02.812805 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:02.812808 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:02.812961 (+ 153us) server_negotiation.cc:407] Connection header received
0806 22:51:02.812997 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:02.812998 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:02.813010 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:02.813067 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:51:02.813261 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:02.813990 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:02.815473 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:02.816025 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:02.816103 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:02.816113 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:02.816390 (+ 277us) server_negotiation.cc:299] Negotiation successful
0806 22:51:02.816433 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 22:51:02.816536 575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:02.812681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.67:38247 (local address 127.6.40.66:34163)
0806 22:51:02.812720 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0806 22:51:02.812734 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 22:51:02.812776 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:51:02.813087 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:51:02.813088 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:51:02.813089 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:51:02.813193 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:02.813195 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:02.814075 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:02.814076 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:02.815375 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:02.815378 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:02.816202 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:02.816203 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:02.816297 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:02.816303 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 22:51:02.816369 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 22:51:02.816425 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18,"mutex_wait_us":63}
I20260806 22:51:02.816917 691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce7f4bfca3ad45108e99539e4e27eaca" candidate_uuid: "33dd8d7dd7694201948c484134dbbe71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28134f440c8a413891363f444a1fa847" is_pre_election: true
I20260806 22:51:02.817085 691 raft_consensus.cc:2454] T ce7f4bfca3ad45108e99539e4e27eaca P 28134f440c8a413891363f444a1fa847 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33dd8d7dd7694201948c484134dbbe71 in term 0.
I20260806 22:51:02.816551 789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:02.812713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.65:35535 (local address 127.6.40.66:39451)
0806 22:51:02.812895 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0806 22:51:02.812905 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:51:02.812954 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:51:02.813087 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:51:02.813087 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:51:02.813089 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:51:02.813193 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:02.813195 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:02.814075 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:02.814076 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:02.815375 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:02.815378 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:02.816216 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:02.816216 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:02.816296 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:02.816303 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 22:51:02.816376 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 22:51:02.816423 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"mutex_wait_us":62,"thread_start_us":95,"threads_started":1}
I20260806 22:51:02.817327 402 leader_election.cc:302] T ce7f4bfca3ad45108e99539e4e27eaca P 33dd8d7dd7694201948c484134dbbe71 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28134f440c8a413891363f444a1fa847, 33dd8d7dd7694201948c484134dbbe71; no voters:
I20260806 22:51:02.817464 784 raft_consensus.cc:2781] T ce7f4bfca3ad45108e99539e4e27eaca P 33dd8d7dd7694201948c484134dbbe71 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:51:02.817523 784 raft_consensus.cc:482] T ce7f4bfca3ad45108e99539e4e27eaca P 33dd8d7dd7694201948c484134dbbe71 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:51:02.817554 784 raft_consensus.cc:3037] T ce7f4bfca3ad45108e99539e4e27eaca P 33dd8d7dd7694201948c484134dbbe71 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:51:02.817580 303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce7f4bfca3ad45108e99539e4e27eaca" candidate_uuid: "33dd8d7dd7694201948c484134dbbe71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75a10f655c446e68bbf59db8d28a839" is_pre_election: true
I20260806 22:51:02.817723 303 raft_consensus.cc:2454] T ce7f4bfca3ad45108e99539e4e27eaca P c75a10f655c446e68bbf59db8d28a839 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33dd8d7dd7694201948c484134dbbe71 in term 0.
I20260806 22:51:02.818374 784 raft_consensus.cc:504] T ce7f4bfca3ad45108e99539e4e27eaca P 33dd8d7dd7694201948c484134dbbe71 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28134f440c8a413891363f444a1fa847" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 38247 } } peers { permanent_uuid: "c75a10f655c446e68bbf59db8d28a839" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 35535 } } peers { permanent_uuid: "33dd8d7dd7694201948c484134dbbe71" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 36449 } }
I20260806 22:51:02.818521 784 leader_election.cc:288] T ce7f4bfca3ad45108e99539e4e27eaca P 33dd8d7dd7694201948c484134dbbe71 [CANDIDATE]: Term 1 election: Requested vote from peers 28134f440c8a413891363f444a1fa847 (127.6.40.67:38247), c75a10f655c446e68bbf59db8d28a839 (127.6.40.65:35535)
I20260806 22:51:02.818655 691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce7f4bfca3ad45108e99539e4e27eaca" candidate_uuid: "33dd8d7dd7694201948c484134dbbe71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28134f440c8a413891363f444a1fa847"
I20260806 22:51:02.818687 303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce7f4bfca3ad45108e99539e4e27eaca" candidate_uuid: "33dd8d7dd7694201948c484134dbbe71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75a10f655c446e68bbf59db8d28a839"
I20260806 22:51:02.818734 691 raft_consensus.cc:3037] T ce7f4bfca3ad45108e99539e4e27eaca P 28134f440c8a413891363f444a1fa847 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:51:02.818749 303 raft_consensus.cc:3037] T ce7f4bfca3ad45108e99539e4e27eaca P c75a10f655c446e68bbf59db8d28a839 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:51:02.819661 691 raft_consensus.cc:2454] T ce7f4bfca3ad45108e99539e4e27eaca P 28134f440c8a413891363f444a1fa847 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33dd8d7dd7694201948c484134dbbe71 in term 1.
I20260806 22:51:02.819661 303 raft_consensus.cc:2454] T ce7f4bfca3ad45108e99539e4e27eaca P c75a10f655c446e68bbf59db8d28a839 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33dd8d7dd7694201948c484134dbbe71 in term 1.
I20260806 22:51:02.819832 403 leader_election.cc:302] T ce7f4bfca3ad45108e99539e4e27eaca P 33dd8d7dd7694201948c484134dbbe71 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33dd8d7dd7694201948c484134dbbe71, c75a10f655c446e68bbf59db8d28a839; no voters:
I20260806 22:51:02.819938 784 raft_consensus.cc:2781] T ce7f4bfca3ad45108e99539e4e27eaca P 33dd8d7dd7694201948c484134dbbe71 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:51:02.820099 784 raft_consensus.cc:686] T ce7f4bfca3ad45108e99539e4e27eaca P 33dd8d7dd7694201948c484134dbbe71 [term 1 LEADER]: Becoming Leader. State: Replica: 33dd8d7dd7694201948c484134dbbe71, State: Running, Role: LEADER
I20260806 22:51:02.820200 784 consensus_queue.cc:237] T ce7f4bfca3ad45108e99539e4e27eaca P 33dd8d7dd7694201948c484134dbbe71 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28134f440c8a413891363f444a1fa847" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 38247 } } peers { permanent_uuid: "c75a10f655c446e68bbf59db8d28a839" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 35535 } } peers { permanent_uuid: "33dd8d7dd7694201948c484134dbbe71" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 36449 } }
I20260806 22:51:02.821053 32551 catalog_manager.cc:5495] T ce7f4bfca3ad45108e99539e4e27eaca P 33dd8d7dd7694201948c484134dbbe71 reported cstate change: term changed from 0 to 1, leader changed from <none> to 33dd8d7dd7694201948c484134dbbe71 (127.6.40.66). New cstate: current_term: 1 leader_uuid: "33dd8d7dd7694201948c484134dbbe71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28134f440c8a413891363f444a1fa847" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 38247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c75a10f655c446e68bbf59db8d28a839" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 35535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33dd8d7dd7694201948c484134dbbe71" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 36449 } health_report { overall_health: HEALTHY } } }
I20260806 22:51:02.827461 769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:02.825322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41040 (local address 127.6.40.66:36449)
0806 22:51:02.825367 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:02.825369 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:02.825376 (+ 7us) server_negotiation.cc:407] Connection header received
0806 22:51:02.825447 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:02.825448 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:02.825465 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:02.825519 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:51:02.825627 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:02.826071 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:02.826782 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:02.826968 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:02.827018 (+ 50us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:02.827147 (+ 129us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:51:02.827226 (+ 79us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:51:02.827292 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:02.827333 (+ 41us) server_negotiation.cc:299] Negotiation successful
0806 22:51:02.827340 (+ 7us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 22:51:02.829104 691 raft_consensus.cc:1261] T ce7f4bfca3ad45108e99539e4e27eaca P 28134f440c8a413891363f444a1fa847 [term 1 FOLLOWER]: Refusing update from remote peer 33dd8d7dd7694201948c484134dbbe71: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:51:02.829115 303 raft_consensus.cc:1261] T ce7f4bfca3ad45108e99539e4e27eaca P c75a10f655c446e68bbf59db8d28a839 [term 1 FOLLOWER]: Refusing update from remote peer 33dd8d7dd7694201948c484134dbbe71: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:51:02.829447 784 consensus_queue.cc:1035] T ce7f4bfca3ad45108e99539e4e27eaca P 33dd8d7dd7694201948c484134dbbe71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c75a10f655c446e68bbf59db8d28a839" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 35535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:51:02.829589 790 consensus_queue.cc:1035] T ce7f4bfca3ad45108e99539e4e27eaca P 33dd8d7dd7694201948c484134dbbe71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28134f440c8a413891363f444a1fa847" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 38247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:51:02.832448 796 mvcc.cc:205] Tried to move back new op lower bound from 7315688090945695744 to 7315688090912342016. Current Snapshot: MvccSnapshot[applied={T|T < 7315688090945695744}]
I20260806 22:51:02.834244 32550 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38110:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:51:02.834342 32550 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38110:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:51:02.835150 32550 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a34a18039d324175bdd1e18ac0d206b7: Sending DeleteTablet for 3 replicas of tablet ce7f4bfca3ad45108e99539e4e27eaca
I20260806 22:51:02.835484 662 tablet_service.cc:1514] Processing DeleteTablet for tablet ce7f4bfca3ad45108e99539e4e27eaca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:51:02 UTC) from {username='slave'} at 127.0.0.1:59448
I20260806 22:51:02.835502 475 tablet_service.cc:1514] Processing DeleteTablet for tablet ce7f4bfca3ad45108e99539e4e27eaca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:51:02 UTC) from {username='slave'} at 127.0.0.1:41036
I20260806 22:51:02.835537 32737 tablet_service.cc:1514] Processing DeleteTablet for tablet ce7f4bfca3ad45108e99539e4e27eaca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:51:02 UTC) from {username='slave'} at 127.0.0.1:36868
I20260806 22:51:02.835770 808 tablet_replica.cc:330] T ce7f4bfca3ad45108e99539e4e27eaca P c75a10f655c446e68bbf59db8d28a839: stopping tablet replica
I20260806 22:51:02.835909 808 raft_consensus.cc:2229] T ce7f4bfca3ad45108e99539e4e27eaca P c75a10f655c446e68bbf59db8d28a839 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:51:02.836017 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 32647
I20260806 22:51:02.836184 808 raft_consensus.cc:2258] T ce7f4bfca3ad45108e99539e4e27eaca P c75a10f655c446e68bbf59db8d28a839 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:51:02.850553 805 tablet_replica.cc:330] T ce7f4bfca3ad45108e99539e4e27eaca P 33dd8d7dd7694201948c484134dbbe71: stopping tablet replica
I20260806 22:51:02.850723 805 raft_consensus.cc:2229] T ce7f4bfca3ad45108e99539e4e27eaca P 33dd8d7dd7694201948c484134dbbe71 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:51:02.850854 805 raft_consensus.cc:2258] T ce7f4bfca3ad45108e99539e4e27eaca P 33dd8d7dd7694201948c484134dbbe71 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:51:02.851243 807 tablet_replica.cc:330] T ce7f4bfca3ad45108e99539e4e27eaca P 28134f440c8a413891363f444a1fa847: stopping tablet replica
I20260806 22:51:02.851378 807 raft_consensus.cc:2229] T ce7f4bfca3ad45108e99539e4e27eaca P 28134f440c8a413891363f444a1fa847 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:51:02.851509 807 raft_consensus.cc:2258] T ce7f4bfca3ad45108e99539e4e27eaca P 28134f440c8a413891363f444a1fa847 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:51:02.852249 805 ts_tablet_manager.cc:1905] T ce7f4bfca3ad45108e99539e4e27eaca P 33dd8d7dd7694201948c484134dbbe71: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:51:02.852563 807 ts_tablet_manager.cc:1905] T ce7f4bfca3ad45108e99539e4e27eaca P 28134f440c8a413891363f444a1fa847: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:51:02.854351 807 ts_tablet_manager.cc:1918] T ce7f4bfca3ad45108e99539e4e27eaca P 28134f440c8a413891363f444a1fa847: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:51:02.854441 807 log.cc:1196] T ce7f4bfca3ad45108e99539e4e27eaca P 28134f440c8a413891363f444a1fa847: Deleting WAL directory at /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056405724609-6305-0/minicluster-data/ts-2/wal/wals/ce7f4bfca3ad45108e99539e4e27eaca
I20260806 22:51:02.854467 805 ts_tablet_manager.cc:1918] T ce7f4bfca3ad45108e99539e4e27eaca P 33dd8d7dd7694201948c484134dbbe71: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:51:02.854555 805 log.cc:1196] T ce7f4bfca3ad45108e99539e4e27eaca P 33dd8d7dd7694201948c484134dbbe71: Deleting WAL directory at /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056405724609-6305-0/minicluster-data/ts-1/wal/wals/ce7f4bfca3ad45108e99539e4e27eaca
I20260806 22:51:02.854722 807 ts_tablet_manager.cc:1939] T ce7f4bfca3ad45108e99539e4e27eaca P 28134f440c8a413891363f444a1fa847: Deleting consensus metadata
I20260806 22:51:02.854868 805 ts_tablet_manager.cc:1939] T ce7f4bfca3ad45108e99539e4e27eaca P 33dd8d7dd7694201948c484134dbbe71: Deleting consensus metadata
I20260806 22:51:02.855212 32530 catalog_manager.cc:4841] TS 28134f440c8a413891363f444a1fa847 (127.6.40.67:38247): tablet ce7f4bfca3ad45108e99539e4e27eaca (table test-table [id=72b8c6d561874ed7ab49204b830ff9ee]) successfully deleted
I20260806 22:51:02.855311 32532 catalog_manager.cc:4841] TS 33dd8d7dd7694201948c484134dbbe71 (127.6.40.66:36449): tablet ce7f4bfca3ad45108e99539e4e27eaca (table test-table [id=72b8c6d561874ed7ab49204b830ff9ee]) successfully deleted
W20260806 22:51:02.857367 32532 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.40.65:35535 (error 108)
I20260806 22:51:02.857910 767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:02.857762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.65:35535 (local address 127.0.0.1:36870)
0806 22:51:02.857826 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0806 22:51:02.857873 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.65:35535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260806 22:51:02.857990 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 381
W20260806 22:51:02.862607 32532 catalog_manager.cc:4584] TS c75a10f655c446e68bbf59db8d28a839 (127.6.40.65:35535): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ce7f4bfca3ad45108e99539e4e27eaca: Network error: Client connection negotiation failed: client connection to 127.6.40.65:35535: connect: Connection refused (error 111)
I20260806 22:51:02.867154 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 576
I20260806 22:51:02.879981 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 32509
2026-08-06T22:51:02Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (718 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:51:02 dist-test-slave-zh2d krb5kdc[812](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:51:02 dist-test-slave-zh2d krb5kdc[812](info): set up 2 sockets
Aug 06 22:51:02 dist-test-slave-zh2d krb5kdc[812](info): commencing operation
krb5kdc: starting...
W20260806 22:51:04.102346 6305 mini_kdc.cc:121] Time spent starting KDC: real 1.196s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/krb5kdc/test-user.keytab.
Aug 06 22:51:04 dist-test-slave-zh2d krb5kdc[812](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056664, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:51:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:51:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.40.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.40.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.40.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab.
Entry for principal kudu/127.6.40.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab.
I20260806 22:51:04.242050 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.126:44517
--webserver_interface=127.6.40.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab
--principal=kudu/127.6.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:46087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.40.126:44517
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:51:04.329761 826 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:51:04.329979 826 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:51:04.330003 826 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:51:04.331173 826 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:51:04.331218 826 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:51:04.331233 826 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:51:04.331244 826 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:51:04.331256 826 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:51:04.332767 826 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:46087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.40.126:44517
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab
--principal=kudu/127.6.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.126:44517
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.826
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:51:04.333038 826 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:51:04.333247 826 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:51:04.338235 839 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:51:04.338235 835 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:51:04.338287 837 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:51:04.338442 826 server_base.cc:1034] running on GCE node
I20260806 22:51:04.338682 826 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:51:04.338990 826 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:51:04.340170 826 hybrid_clock.cc:648] HybridClock initialized: now 1786056664340148 us; error 40 us; skew 500 ppm
Aug 06 22:51:04 dist-test-slave-zh2d krb5kdc[812](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056664, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:51:04.342826 826 init.cc:377] Logged in from keytab as kudu/127.6.40.126@KRBTEST.COM (short username kudu)
I20260806 22:51:04.343312 826 webserver.cc:460] Webserver started at http://127.6.40.126:45847/ using document root <none> and password file <none>
I20260806 22:51:04.343564 826 fs_manager.cc:359] Metadata directory not provided
I20260806 22:51:04.343632 826 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:51:04.343775 826 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:51:04.344894 826 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/minicluster-data/master-0/data/instance:
uuid: "ae4c8b0f0aa141efab70e26d501c8c6f"
format_stamp: "Formatted at 2026-08-06 22:51:04 on dist-test-slave-zh2d"
server_key: "f2e49cc78efa259d6e8e6f6db0362c0a"
server_key_iv: "c8405155e09673d98aead78b05e0ebf7"
server_key_version: "encryptionkey@0"
I20260806 22:51:04.345283 826 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/minicluster-data/master-0/wal/instance:
uuid: "ae4c8b0f0aa141efab70e26d501c8c6f"
format_stamp: "Formatted at 2026-08-06 22:51:04 on dist-test-slave-zh2d"
server_key: "f2e49cc78efa259d6e8e6f6db0362c0a"
server_key_iv: "c8405155e09673d98aead78b05e0ebf7"
server_key_version: "encryptionkey@0"
I20260806 22:51:04.346648 826 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:51:04.347440 847 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:51:04.347658 826 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:51:04.347749 826 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/minicluster-data/master-0/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/minicluster-data/master-0/wal
uuid: "ae4c8b0f0aa141efab70e26d501c8c6f"
format_stamp: "Formatted at 2026-08-06 22:51:04 on dist-test-slave-zh2d"
server_key: "f2e49cc78efa259d6e8e6f6db0362c0a"
server_key_iv: "c8405155e09673d98aead78b05e0ebf7"
server_key_version: "encryptionkey@0"
I20260806 22:51:04.347803 826 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:51:04.357264 826 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:51:04.359503 826 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:51:04.359619 826 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:51:04.364033 826 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.126:44517
I20260806 22:51:04.364409 826 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
I20260806 22:51:04.364389 923 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.126:44517 every 8 connection(s)
I20260806 22:51:04.365221 924 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:51:04.367646 924 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae4c8b0f0aa141efab70e26d501c8c6f: Bootstrap starting.
I20260806 22:51:04.367918 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 826
I20260806 22:51:04.368005 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/minicluster-data/master-0/wal/instance
I20260806 22:51:04.368214 6305 external_mini_cluster.cc:1428] Setting key d8ceb6eda4d00fb744a445479a1c0620
I20260806 22:51:04.368371 924 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ae4c8b0f0aa141efab70e26d501c8c6f: Neither blocks nor log segments found. Creating new log.
I20260806 22:51:04.368767 924 log.cc:824] T 00000000000000000000000000000000 P ae4c8b0f0aa141efab70e26d501c8c6f: Log is configured to *not* fsync() on all Append() calls
I20260806 22:51:04.369498 924 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae4c8b0f0aa141efab70e26d501c8c6f: No bootstrap required, opened a new log
I20260806 22:51:04.371348 924 raft_consensus.cc:348] T 00000000000000000000000000000000 P ae4c8b0f0aa141efab70e26d501c8c6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae4c8b0f0aa141efab70e26d501c8c6f" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 44517 } }
I20260806 22:51:04.371505 924 raft_consensus.cc:374] T 00000000000000000000000000000000 P ae4c8b0f0aa141efab70e26d501c8c6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:51:04.371553 924 raft_consensus.cc:729] T 00000000000000000000000000000000 P ae4c8b0f0aa141efab70e26d501c8c6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae4c8b0f0aa141efab70e26d501c8c6f, State: Initialized, Role: FOLLOWER
I20260806 22:51:04.371703 924 consensus_queue.cc:260] T 00000000000000000000000000000000 P ae4c8b0f0aa141efab70e26d501c8c6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae4c8b0f0aa141efab70e26d501c8c6f" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 44517 } }
I20260806 22:51:04.371798 924 raft_consensus.cc:388] T 00000000000000000000000000000000 P ae4c8b0f0aa141efab70e26d501c8c6f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:51:04.371850 924 raft_consensus.cc:482] T 00000000000000000000000000000000 P ae4c8b0f0aa141efab70e26d501c8c6f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:51:04.371898 924 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ae4c8b0f0aa141efab70e26d501c8c6f [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:51:04.372443 924 raft_consensus.cc:504] T 00000000000000000000000000000000 P ae4c8b0f0aa141efab70e26d501c8c6f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae4c8b0f0aa141efab70e26d501c8c6f" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 44517 } }
I20260806 22:51:04.372567 924 leader_election.cc:302] T 00000000000000000000000000000000 P ae4c8b0f0aa141efab70e26d501c8c6f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ae4c8b0f0aa141efab70e26d501c8c6f; no voters:
I20260806 22:51:04.372764 924 leader_election.cc:288] T 00000000000000000000000000000000 P ae4c8b0f0aa141efab70e26d501c8c6f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:51:04.372792 930 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ae4c8b0f0aa141efab70e26d501c8c6f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:51:04.372995 930 raft_consensus.cc:686] T 00000000000000000000000000000000 P ae4c8b0f0aa141efab70e26d501c8c6f [term 1 LEADER]: Becoming Leader. State: Replica: ae4c8b0f0aa141efab70e26d501c8c6f, State: Running, Role: LEADER
I20260806 22:51:04.373142 924 sys_catalog.cc:564] T 00000000000000000000000000000000 P ae4c8b0f0aa141efab70e26d501c8c6f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:51:04.373153 930 consensus_queue.cc:237] T 00000000000000000000000000000000 P ae4c8b0f0aa141efab70e26d501c8c6f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae4c8b0f0aa141efab70e26d501c8c6f" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 44517 } }
I20260806 22:51:04.373636 931 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae4c8b0f0aa141efab70e26d501c8c6f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ae4c8b0f0aa141efab70e26d501c8c6f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae4c8b0f0aa141efab70e26d501c8c6f" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 44517 } } }
I20260806 22:51:04.373808 931 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae4c8b0f0aa141efab70e26d501c8c6f [sys.catalog]: This master's current role is: LEADER
I20260806 22:51:04.374562 943 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:51:04.375084 943 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 06 22:51:04 dist-test-slave-zh2d krb5kdc[812](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056664, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
I20260806 22:51:04.375933 933 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae4c8b0f0aa141efab70e26d501c8c6f [sys.catalog]: SysCatalogTable state changed. Reason: New leader ae4c8b0f0aa141efab70e26d501c8c6f. Latest consensus state: current_term: 1 leader_uuid: "ae4c8b0f0aa141efab70e26d501c8c6f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae4c8b0f0aa141efab70e26d501c8c6f" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 44517 } } }
I20260806 22:51:04.376036 933 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae4c8b0f0aa141efab70e26d501c8c6f [sys.catalog]: This master's current role is: LEADER
I20260806 22:51:04.376675 943 catalog_manager.cc:1347] Generated new cluster ID: 34c57caa772e4add8c5e0848cc92065f
I20260806 22:51:04.376729 943 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:51:04.379626 928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:04.368747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58568 (local address 127.6.40.126:44517)
0806 22:51:04.369013 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:04.369022 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:04.369052 (+ 30us) server_negotiation.cc:407] Connection header received
0806 22:51:04.369523 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:04.369531 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:04.369592 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:04.369730 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:04.370391 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:04.371270 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:04.372237 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:04.372743 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:04.372874 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:04.376210 (+ 3336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:51:04.376215 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:51:04.376215 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:51:04.376228 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:51:04.378044 (+ 1816us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:04.378420 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:04.378423 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:04.378424 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:04.378528 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:04.378742 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:04.378743 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:04.378743 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:04.378978 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:51:04.379150 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:04.379256 (+ 106us) server_negotiation.cc:299] Negotiation successful
0806 22:51:04.379396 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":218,"thread_start_us":76,"threads_started":1}
I20260806 22:51:04.392583 943 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:51:04.393373 943 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:51:04.401835 943 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ae4c8b0f0aa141efab70e26d501c8c6f: Generated new TSK 0
I20260806 22:51:04.402041 943 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.40.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.40.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.40.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.65.keytab.
Entry for principal kudu/127.6.40.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.65.keytab.
I20260806 22:51:04.436259 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.65:0
--local_ip_for_outbound_sockets=127.6.40.65
--webserver_interface=127.6.40.65
--webserver_port=0
--tserver_master_addrs=127.6.40.126:44517
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.65.keytab
--principal=kudu/127.6.40.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:46087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:51:04.521013 956 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:51:04.521216 956 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:51:04.521239 956 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:51:04.522398 956 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:51:04.522440 956 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:51:04.522467 956 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.65
I20260806 22:51:04.523924 956 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:46087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.65.keytab
--principal=kudu/127.6.40.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.40.65
--webserver_port=0
--tserver_master_addrs=127.6.40.126:44517
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.956
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.65
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:51:04.524184 956 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:51:04.524439 956 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:51:04.529718 962 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:51:04.529691 966 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:51:04.529716 963 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:51:04.530108 956 server_base.cc:1034] running on GCE node
I20260806 22:51:04.530314 956 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:51:04.530516 956 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:51:04.531657 956 hybrid_clock.cc:648] HybridClock initialized: now 1786056664531646 us; error 30 us; skew 500 ppm
Aug 06 22:51:04 dist-test-slave-zh2d krb5kdc[812](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056664, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:51:04.534368 956 init.cc:377] Logged in from keytab as kudu/127.6.40.65@KRBTEST.COM (short username kudu)
I20260806 22:51:04.534876 956 webserver.cc:460] Webserver started at http://127.6.40.65:46315/ using document root <none> and password file <none>
I20260806 22:51:04.535142 956 fs_manager.cc:359] Metadata directory not provided
I20260806 22:51:04.535213 956 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:51:04.535364 956 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:51:04.536435 956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/minicluster-data/ts-0/data/instance:
uuid: "9272551bb766498dbb8924f72a650d6c"
format_stamp: "Formatted at 2026-08-06 22:51:04 on dist-test-slave-zh2d"
server_key: "f246a39e568db5037d3806a455962b11"
server_key_iv: "46b73e238d08a70812fcabc62b4e6206"
server_key_version: "encryptionkey@0"
I20260806 22:51:04.536815 956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/minicluster-data/ts-0/wal/instance:
uuid: "9272551bb766498dbb8924f72a650d6c"
format_stamp: "Formatted at 2026-08-06 22:51:04 on dist-test-slave-zh2d"
server_key: "f246a39e568db5037d3806a455962b11"
server_key_iv: "46b73e238d08a70812fcabc62b4e6206"
server_key_version: "encryptionkey@0"
I20260806 22:51:04.538194 956 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:51:04.538929 980 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:51:04.539099 956 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:51:04.539206 956 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/minicluster-data/ts-0/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/minicluster-data/ts-0/wal
uuid: "9272551bb766498dbb8924f72a650d6c"
format_stamp: "Formatted at 2026-08-06 22:51:04 on dist-test-slave-zh2d"
server_key: "f246a39e568db5037d3806a455962b11"
server_key_iv: "46b73e238d08a70812fcabc62b4e6206"
server_key_version: "encryptionkey@0"
I20260806 22:51:04.539280 956 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:51:04.559216 956 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:51:04.561435 956 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:51:04.561619 956 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:51:04.561941 956 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:51:04.562326 956 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:51:04.562389 956 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:51:04.562440 956 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:51:04.562470 956 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:51:04.569229 956 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.65:38043
I20260806 22:51:04.569322 1131 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.65:38043 every 8 connection(s)
I20260806 22:51:04.569680 956 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
I20260806 22:51:04.572232 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 956
I20260806 22:51:04.572381 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/minicluster-data/ts-0/wal/instance
I20260806 22:51:04.572626 6305 external_mini_cluster.cc:1428] Setting key d86c89b47ca79f2957122c8e7fbc013b
Aug 06 22:51:04 dist-test-slave-zh2d krb5kdc[812](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056664, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.65@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
I20260806 22:51:04.581095 928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:04.570564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.65:55435 (local address 127.6.40.126:44517)
0806 22:51:04.570621 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:04.570623 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:04.570852 (+ 229us) server_negotiation.cc:407] Connection header received
0806 22:51:04.571372 (+ 520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:04.571373 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:04.571383 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:04.571444 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:04.572835 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:04.573467 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:04.574626 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:04.574845 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:04.574918 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:04.577680 (+ 2762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:51:04.577681 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:51:04.577682 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:51:04.577691 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:51:04.579494 (+ 1803us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:04.579944 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:04.579946 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:04.579946 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:04.580028 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:04.580305 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:04.580306 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:04.580306 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:04.580465 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:51:04.580546 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:04.580870 (+ 324us) server_negotiation.cc:299] Negotiation successful
0806 22:51:04.580973 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 22:51:04.581444 1134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:04.570519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:44517 (local address 127.6.40.65:55435)
0806 22:51:04.570700 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0806 22:51:04.570750 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 22:51:04.571284 (+ 534us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:51:04.571504 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:51:04.571507 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:51:04.572310 (+ 803us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:51:04.572654 (+ 344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:04.572662 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:04.573586 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:04.573587 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:04.574524 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:04.574527 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:04.574943 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:04.574944 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:04.575081 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:04.575668 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:51:04.575678 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:51:04.577572 (+ 1894us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:51:04.579589 (+ 2017us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:51:04.579596 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:51:04.579598 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:51:04.579858 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:51:04.580114 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:51:04.580115 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:51:04.580115 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:51:04.580230 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:51:04.580555 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:51:04.580561 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:51:04.580697 (+ 136us) client_negotiation.cc:769] Sending connection context
0806 22:51:04.580797 (+ 100us) client_negotiation.cc:240] Negotiation successful
0806 22:51:04.581117 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":104,"threads_started":1}
I20260806 22:51:04.582077 1132 heartbeater.cc:347] Connected to a master server at 127.6.40.126:44517
I20260806 22:51:04.582198 1132 heartbeater.cc:464] Registering TS with master...
I20260806 22:51:04.582456 1132 heartbeater.cc:511] Master 127.6.40.126:44517 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.6.40.66@KRBTEST.COM; defaulting to no policy
I20260806 22:51:04.583153 874 ts_manager.cc:194] Registered new tserver with Master: 9272551bb766498dbb8924f72a650d6c (127.6.40.65:38043)
I20260806 22:51:04.583817 874 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.40.65@KRBTEST.COM'} at 127.6.40.65:55435
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.40.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.40.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.66.keytab.
Entry for principal kudu/127.6.40.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.66.keytab.
I20260806 22:51:04.603636 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.66:0
--local_ip_for_outbound_sockets=127.6.40.66
--webserver_interface=127.6.40.66
--webserver_port=0
--tserver_master_addrs=127.6.40.126:44517
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.66.keytab
--principal=kudu/127.6.40.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:46087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:51:04.687803 1139 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:51:04.688053 1139 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:51:04.688092 1139 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:51:04.689286 1139 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:51:04.689354 1139 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:51:04.689397 1139 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.66
I20260806 22:51:04.690884 1139 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:46087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.66.keytab
--principal=kudu/127.6.40.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.40.66
--webserver_port=0
--tserver_master_addrs=127.6.40.126:44517
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1139
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.66
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:51:04.691169 1139 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:51:04.691411 1139 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:51:04.696899 1150 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:51:04.696936 1139 server_base.cc:1034] running on GCE node
W20260806 22:51:04.696920 1146 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:51:04.696955 1145 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:51:04.697372 1139 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:51:04.697638 1139 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:51:04.698803 1139 hybrid_clock.cc:648] HybridClock initialized: now 1786056664698791 us; error 45 us; skew 500 ppm
Aug 06 22:51:04 dist-test-slave-zh2d krb5kdc[812](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056664, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:51:04.701409 1139 init.cc:377] Logged in from keytab as kudu/127.6.40.66@KRBTEST.COM (short username kudu)
I20260806 22:51:04.701900 1139 webserver.cc:460] Webserver started at http://127.6.40.66:36831/ using document root <none> and password file <none>
I20260806 22:51:04.702157 1139 fs_manager.cc:359] Metadata directory not provided
I20260806 22:51:04.702225 1139 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:51:04.702375 1139 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:51:04.703425 1139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/minicluster-data/ts-1/data/instance:
uuid: "9f631ff72c7247ba8464150a716d2359"
format_stamp: "Formatted at 2026-08-06 22:51:04 on dist-test-slave-zh2d"
server_key: "786eb48023fd142aeb33c65cd788b782"
server_key_iv: "199d026a189cd3875a123d96a273cd69"
server_key_version: "encryptionkey@0"
I20260806 22:51:04.703784 1139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/minicluster-data/ts-1/wal/instance:
uuid: "9f631ff72c7247ba8464150a716d2359"
format_stamp: "Formatted at 2026-08-06 22:51:04 on dist-test-slave-zh2d"
server_key: "786eb48023fd142aeb33c65cd788b782"
server_key_iv: "199d026a189cd3875a123d96a273cd69"
server_key_version: "encryptionkey@0"
I20260806 22:51:04.705193 1139 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:51:04.705961 1160 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:51:04.706147 1139 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:51:04.706254 1139 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/minicluster-data/ts-1/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/minicluster-data/ts-1/wal
uuid: "9f631ff72c7247ba8464150a716d2359"
format_stamp: "Formatted at 2026-08-06 22:51:04 on dist-test-slave-zh2d"
server_key: "786eb48023fd142aeb33c65cd788b782"
server_key_iv: "199d026a189cd3875a123d96a273cd69"
server_key_version: "encryptionkey@0"
I20260806 22:51:04.706328 1139 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:51:04.732359 1139 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:51:04.734445 1139 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:51:04.734634 1139 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:51:04.734915 1139 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:51:04.735266 1139 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:51:04.735322 1139 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:51:04.735357 1139 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:51:04.735386 1139 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:51:04.741931 1139 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.66:37329
I20260806 22:51:04.741990 1326 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.66:37329 every 8 connection(s)
I20260806 22:51:04.742353 1139 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
Aug 06 22:51:04 dist-test-slave-zh2d krb5kdc[812](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056664, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.66@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
I20260806 22:51:04.749868 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 1139
I20260806 22:51:04.750007 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/minicluster-data/ts-1/wal/instance
I20260806 22:51:04.750265 6305 external_mini_cluster.cc:1428] Setting key 52449eaa09d73e00c119ec76fda29da8
I20260806 22:51:04.752108 928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:04.743138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.66:52229 (local address 127.6.40.126:44517)
0806 22:51:04.743209 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:04.743211 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:04.743416 (+ 205us) server_negotiation.cc:407] Connection header received
0806 22:51:04.743977 (+ 561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:04.743978 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:04.743987 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:04.744045 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:04.745050 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:04.745547 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:04.746586 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:04.746765 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:04.746816 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:04.748983 (+ 2167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:51:04.748985 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:51:04.748985 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:51:04.748992 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:51:04.750493 (+ 1501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:04.751003 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:04.751005 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:04.751005 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:04.751077 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:04.751390 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:04.751391 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:04.751392 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:04.751544 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:51:04.751618 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:04.751871 (+ 253us) server_negotiation.cc:299] Negotiation successful
0806 22:51:04.751967 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 22:51:04.752128 1331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:04.743080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:44517 (local address 127.6.40.66:52229)
0806 22:51:04.743271 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0806 22:51:04.743314 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 22:51:04.743870 (+ 556us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:51:04.744110 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:51:04.744113 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:51:04.744712 (+ 599us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:51:04.744967 (+ 255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:04.744972 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:04.745643 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:04.745644 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:04.746500 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:04.746503 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:04.746849 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:04.746850 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:04.746954 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:04.747438 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:51:04.747447 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:51:04.748873 (+ 1426us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:51:04.750603 (+ 1730us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:51:04.750609 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:51:04.750611 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:51:04.750902 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:51:04.751160 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:51:04.751161 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:51:04.751162 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:51:04.751273 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:51:04.751614 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:51:04.751620 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:51:04.751739 (+ 119us) client_negotiation.cc:769] Sending connection context
0806 22:51:04.751825 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 22:51:04.751952 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":113,"threads_started":1}
I20260806 22:51:04.752776 1328 heartbeater.cc:347] Connected to a master server at 127.6.40.126:44517
I20260806 22:51:04.752900 1328 heartbeater.cc:464] Registering TS with master...
I20260806 22:51:04.753096 1328 heartbeater.cc:511] Master 127.6.40.126:44517 requested a full tablet report, sending...
I20260806 22:51:04.753530 874 ts_manager.cc:194] Registered new tserver with Master: 9f631ff72c7247ba8464150a716d2359 (127.6.40.66:37329)
I20260806 22:51:04.753902 874 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.40.66@KRBTEST.COM'} at 127.6.40.66:52229
WARNING: no policy specified for kudu/127.6.40.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.40.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.40.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.67.keytab.
Entry for principal kudu/127.6.40.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.67.keytab.
I20260806 22:51:04.777731 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.67:0
--local_ip_for_outbound_sockets=127.6.40.67
--webserver_interface=127.6.40.67
--webserver_port=0
--tserver_master_addrs=127.6.40.126:44517
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.67.keytab
--principal=kudu/127.6.40.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:46087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:51:04.863160 1335 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:51:04.863368 1335 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:51:04.863391 1335 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:51:04.864550 1335 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:51:04.864594 1335 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:51:04.864650 1335 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.67
I20260806 22:51:04.866168 1335 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:46087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.67.keytab
--principal=kudu/127.6.40.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.40.67
--webserver_port=0
--tserver_master_addrs=127.6.40.126:44517
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1335
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.67
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:51:04.866415 1335 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:51:04.866655 1335 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:51:04.872061 1344 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:51:04.872028 1341 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:51:04.872020 1342 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:51:04.872179 1335 server_base.cc:1034] running on GCE node
I20260806 22:51:04.872555 1335 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:51:04.872825 1335 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:51:04.874046 1335 hybrid_clock.cc:648] HybridClock initialized: now 1786056664874037 us; error 38 us; skew 500 ppm
Aug 06 22:51:04 dist-test-slave-zh2d krb5kdc[812](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056664, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:51:04.876713 1335 init.cc:377] Logged in from keytab as kudu/127.6.40.67@KRBTEST.COM (short username kudu)
I20260806 22:51:04.877202 1335 webserver.cc:460] Webserver started at http://127.6.40.67:33267/ using document root <none> and password file <none>
I20260806 22:51:04.877439 1335 fs_manager.cc:359] Metadata directory not provided
I20260806 22:51:04.877507 1335 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:51:04.877666 1335 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:51:04.878767 1335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/minicluster-data/ts-2/data/instance:
uuid: "bdbabcae759943ac8a6e7761e40de524"
format_stamp: "Formatted at 2026-08-06 22:51:04 on dist-test-slave-zh2d"
server_key: "97ab7954ee99d155d47b5731d154f200"
server_key_iv: "1bf6d26e3bd4cac50091aee22821d41b"
server_key_version: "encryptionkey@0"
I20260806 22:51:04.879231 1335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/minicluster-data/ts-2/wal/instance:
uuid: "bdbabcae759943ac8a6e7761e40de524"
format_stamp: "Formatted at 2026-08-06 22:51:04 on dist-test-slave-zh2d"
server_key: "97ab7954ee99d155d47b5731d154f200"
server_key_iv: "1bf6d26e3bd4cac50091aee22821d41b"
server_key_version: "encryptionkey@0"
I20260806 22:51:04.880633 1335 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:51:04.881541 1356 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:51:04.881754 1335 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:51:04.881870 1335 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/minicluster-data/ts-2/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/minicluster-data/ts-2/wal
uuid: "bdbabcae759943ac8a6e7761e40de524"
format_stamp: "Formatted at 2026-08-06 22:51:04 on dist-test-slave-zh2d"
server_key: "97ab7954ee99d155d47b5731d154f200"
server_key_iv: "1bf6d26e3bd4cac50091aee22821d41b"
server_key_version: "encryptionkey@0"
I20260806 22:51:04.881948 1335 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:51:04.898751 1335 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:51:04.900820 1335 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:51:04.901010 1335 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:51:04.901223 1335 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:51:04.901481 1335 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:51:04.901518 1335 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:51:04.901540 1335 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:51:04.901577 1335 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:51:04.907732 1335 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.67:46143
I20260806 22:51:04.907833 1508 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.67:46143 every 8 connection(s)
I20260806 22:51:04.908142 1335 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
I20260806 22:51:04.913391 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 1335
I20260806 22:51:04.913516 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/minicluster-data/ts-2/wal/instance
I20260806 22:51:04.913759 6305 external_mini_cluster.cc:1428] Setting key bd81537ec4b3fb7ffe517d1bfb7ed82a
Aug 06 22:51:04 dist-test-slave-zh2d krb5kdc[812](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056664, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.67@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
I20260806 22:51:04.918244 928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:04.908951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.67:37611 (local address 127.6.40.126:44517)
0806 22:51:04.909016 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:04.909018 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:04.909300 (+ 282us) server_negotiation.cc:407] Connection header received
0806 22:51:04.909831 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:04.909832 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:04.909843 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:04.909897 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:04.910869 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:04.911364 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:04.912458 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:04.912641 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:04.912694 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:04.915431 (+ 2737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:51:04.915432 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:51:04.915433 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:51:04.915440 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:51:04.916961 (+ 1521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:04.917417 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:04.917418 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:04.917419 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:04.917472 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:04.917680 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:04.917680 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:04.917681 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:04.917779 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:51:04.917828 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:04.918112 (+ 284us) server_negotiation.cc:299] Negotiation successful
0806 22:51:04.918177 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 22:51:04.918589 1515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:04.908957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:44517 (local address 127.6.40.67:37611)
0806 22:51:04.909169 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0806 22:51:04.909208 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 22:51:04.909741 (+ 533us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:51:04.909954 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:51:04.909973 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:51:04.910586 (+ 613us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:51:04.910788 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:04.910794 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:04.911471 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:04.911472 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:04.912376 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:04.912379 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:04.912744 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:04.912745 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:04.912974 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:04.913549 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:51:04.913561 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:51:04.915319 (+ 1758us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:51:04.917068 (+ 1749us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:51:04.917076 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:51:04.917078 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:51:04.917344 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:51:04.917527 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:51:04.917528 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:51:04.917528 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:51:04.917633 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:51:04.917843 (+ 210us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:51:04.917847 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:51:04.917991 (+ 144us) client_negotiation.cc:769] Sending connection context
0806 22:51:04.918088 (+ 97us) client_negotiation.cc:240] Negotiation successful
0806 22:51:04.918265 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":105,"threads_started":1}
I20260806 22:51:04.919180 1511 heartbeater.cc:347] Connected to a master server at 127.6.40.126:44517
I20260806 22:51:04.919298 1511 heartbeater.cc:464] Registering TS with master...
I20260806 22:51:04.919548 1511 heartbeater.cc:511] Master 127.6.40.126:44517 requested a full tablet report, sending...
I20260806 22:51:04.919966 874 ts_manager.cc:194] Registered new tserver with Master: bdbabcae759943ac8a6e7761e40de524 (127.6.40.67:46143)
I20260806 22:51:04.920385 874 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.40.67@KRBTEST.COM'} at 127.6.40.67:37611
I20260806 22:51:04.925793 6305 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:51:04.933929 928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:04.927292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58574 (local address 127.6.40.126:44517)
0806 22:51:04.927375 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:04.927377 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:04.927527 (+ 150us) server_negotiation.cc:407] Connection header received
0806 22:51:04.927609 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:04.927610 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:04.927620 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:04.927694 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:04.928307 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:04.928896 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:04.929714 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:04.929945 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:04.930040 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:04.931327 (+ 1287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:51:04.931328 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:51:04.931329 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:51:04.931337 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:51:04.932611 (+ 1274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:04.933066 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:04.933067 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:04.933067 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:04.933129 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:04.933375 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:04.933376 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:04.933376 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:04.933515 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:51:04.933624 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:04.933751 (+ 127us) server_negotiation.cc:299] Negotiation successful
0806 22:51:04.933827 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 22:51:04.935464 874 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58574:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:51:04.937069 874 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:51:04.943778 1530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:04.939353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.66:37329 (local address 127.0.0.1:39426)
0806 22:51:04.939528 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0806 22:51:04.939591 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0806 22:51:04.939663 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:51:04.940121 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:51:04.940124 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:51:04.940127 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:51:04.940198 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:04.940200 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:04.941297 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:04.941298 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:04.942550 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:04.942552 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:04.943449 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:04.943450 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:04.943530 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:04.943544 (+ 14us) client_negotiation.cc:769] Sending connection context
0806 22:51:04.943626 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 22:51:04.943690 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"mutex_wait_us":64,"thread_start_us":63,"threads_started":1}
I20260806 22:51:04.943796 1532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:04.939499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39426 (local address 127.6.40.66:37329)
0806 22:51:04.939964 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:04.939968 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:04.939978 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:51:04.940014 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:04.940015 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:04.940032 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:04.940111 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:51:04.940294 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:04.941194 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:04.942644 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:04.943355 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:04.943432 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:04.943488 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:04.943658 (+ 170us) server_negotiation.cc:299] Negotiation successful
0806 22:51:04.943705 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":430,"thread_start_us":107,"threads_started":1}
I20260806 22:51:04.943925 1529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:04.939353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.67:46143 (local address 127.0.0.1:37326)
0806 22:51:04.939714 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0806 22:51:04.939724 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:51:04.939777 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:51:04.940138 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:51:04.940139 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:51:04.940142 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:51:04.940213 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:04.940216 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:04.941631 (+ 1415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:04.941632 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:04.942766 (+ 1134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:04.942769 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:04.943687 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:04.943689 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:04.943753 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:04.943758 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:51:04.943790 (+ 32us) client_negotiation.cc:240] Negotiation successful
0806 22:51:04.943836 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":347,"mutex_wait_us":4,"thread_start_us":109,"threads_started":1}
I20260806 22:51:04.944038 1533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:04.939585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.65:38043 (local address 127.0.0.1:44180)
0806 22:51:04.940210 (+ 625us) negotiation.cc:107] Waiting for socket to connect
0806 22:51:04.940220 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:51:04.940268 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:51:04.940500 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:51:04.940501 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:51:04.940503 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:51:04.940538 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:04.940540 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:04.941789 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:04.941790 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:04.942885 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:04.942887 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:04.943774 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:04.943776 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:04.943840 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:04.943845 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:51:04.943911 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 22:51:04.943959 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":606,"mutex_wait_us":1,"thread_start_us":68,"threads_started":1}
I20260806 22:51:04.944571 1535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:04.939724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37326 (local address 127.6.40.67:46143)
0806 22:51:04.939907 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:04.939912 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:04.939923 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:51:04.939965 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:04.939968 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:04.939986 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:04.940301 (+ 315us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:51:04.940381 (+ 80us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:04.941531 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:04.942856 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:04.943571 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:04.944339 (+ 768us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:04.944407 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:04.944433 (+ 26us) server_negotiation.cc:299] Negotiation successful
0806 22:51:04.944478 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":103,"threads_started":1}
I20260806 22:51:04.944881 1534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:04.939732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44180 (local address 127.6.40.65:38043)
0806 22:51:04.939924 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:04.939929 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:04.940333 (+ 404us) server_negotiation.cc:407] Connection header received
0806 22:51:04.940384 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:04.940386 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:04.940402 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:04.940486 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:51:04.940607 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:04.941688 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:04.942979 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:04.943625 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:04.943687 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:04.943740 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:04.944156 (+ 416us) server_negotiation.cc:299] Negotiation successful
0806 22:51:04.944205 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":113,"threads_started":1}
I20260806 22:51:04.944737 1234 tablet_service.cc:1467] Processing CreateTablet for tablet 7dcc4cd7fd304dcfab6ac4f12b65f3b9 (DEFAULT_TABLE table=test-table [id=5310dc00b92a46deb24b75c65342e383]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:51:04.945168 1234 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7dcc4cd7fd304dcfab6ac4f12b65f3b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:51:04.945271 1423 tablet_service.cc:1467] Processing CreateTablet for tablet 7dcc4cd7fd304dcfab6ac4f12b65f3b9 (DEFAULT_TABLE table=test-table [id=5310dc00b92a46deb24b75c65342e383]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:51:04.945600 1423 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7dcc4cd7fd304dcfab6ac4f12b65f3b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:51:04.945525 1045 tablet_service.cc:1467] Processing CreateTablet for tablet 7dcc4cd7fd304dcfab6ac4f12b65f3b9 (DEFAULT_TABLE table=test-table [id=5310dc00b92a46deb24b75c65342e383]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:51:04.945849 1045 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7dcc4cd7fd304dcfab6ac4f12b65f3b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:51:04.948410 1538 tablet_bootstrap.cc:492] T 7dcc4cd7fd304dcfab6ac4f12b65f3b9 P 9272551bb766498dbb8924f72a650d6c: Bootstrap starting.
I20260806 22:51:04.948982 1540 tablet_bootstrap.cc:492] T 7dcc4cd7fd304dcfab6ac4f12b65f3b9 P 9f631ff72c7247ba8464150a716d2359: Bootstrap starting.
I20260806 22:51:04.949231 1538 tablet_bootstrap.cc:654] T 7dcc4cd7fd304dcfab6ac4f12b65f3b9 P 9272551bb766498dbb8924f72a650d6c: Neither blocks nor log segments found. Creating new log.
I20260806 22:51:04.949589 1540 tablet_bootstrap.cc:654] T 7dcc4cd7fd304dcfab6ac4f12b65f3b9 P 9f631ff72c7247ba8464150a716d2359: Neither blocks nor log segments found. Creating new log.
I20260806 22:51:04.949776 1538 log.cc:824] T 7dcc4cd7fd304dcfab6ac4f12b65f3b9 P 9272551bb766498dbb8924f72a650d6c: Log is configured to *not* fsync() on all Append() calls
I20260806 22:51:04.950155 1543 tablet_bootstrap.cc:492] T 7dcc4cd7fd304dcfab6ac4f12b65f3b9 P bdbabcae759943ac8a6e7761e40de524: Bootstrap starting.
I20260806 22:51:04.950279 1540 log.cc:824] T 7dcc4cd7fd304dcfab6ac4f12b65f3b9 P 9f631ff72c7247ba8464150a716d2359: Log is configured to *not* fsync() on all Append() calls
I20260806 22:51:04.950728 1538 tablet_bootstrap.cc:492] T 7dcc4cd7fd304dcfab6ac4f12b65f3b9 P 9272551bb766498dbb8924f72a650d6c: No bootstrap required, opened a new log
I20260806 22:51:04.950850 1543 tablet_bootstrap.cc:654] T 7dcc4cd7fd304dcfab6ac4f12b65f3b9 P bdbabcae759943ac8a6e7761e40de524: Neither blocks nor log segments found. Creating new log.
I20260806 22:51:04.950843 1538 ts_tablet_manager.cc:1397] T 7dcc4cd7fd304dcfab6ac4f12b65f3b9 P 9272551bb766498dbb8924f72a650d6c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:51:04.951061 1540 tablet_bootstrap.cc:492] T 7dcc4cd7fd304dcfab6ac4f12b65f3b9 P 9f631ff72c7247ba8464150a716d2359: No bootstrap required, opened a new log
I20260806 22:51:04.951143 1540 ts_tablet_manager.cc:1397] T 7dcc4cd7fd304dcfab6ac4f12b65f3b9 P 9f631ff72c7247ba8464150a716d2359: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:51:04.951409 1543 log.cc:824] T 7dcc4cd7fd304dcfab6ac4f12b65f3b9 P bdbabcae759943ac8a6e7761e40de524: Log is configured to *not* fsync() on all Append() calls
I20260806 22:51:04.952215 1543 tablet_bootstrap.cc:492] T 7dcc4cd7fd304dcfab6ac4f12b65f3b9 P bdbabcae759943ac8a6e7761e40de524: No bootstrap required, opened a new log
I20260806 22:51:04.952279 1543 ts_tablet_manager.cc:1397] T 7dcc4cd7fd304dcfab6ac4f12b65f3b9 P bdbabcae759943ac8a6e7761e40de524: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:51:04.953063 1540 raft_consensus.cc:348] T 7dcc4cd7fd304dcfab6ac4f12b65f3b9 P 9f631ff72c7247ba8464150a716d2359 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdbabcae759943ac8a6e7761e40de524" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 46143 } } peers { permanent_uuid: "9f631ff72c7247ba8464150a716d2359" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 37329 } } peers { permanent_uuid: "9272551bb766498dbb8924f72a650d6c" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 38043 } }
I20260806 22:51:04.953231 1540 raft_consensus.cc:374] T 7dcc4cd7fd304dcfab6ac4f12b65f3b9 P 9f631ff72c7247ba8464150a716d2359 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:51:04.953276 1540 raft_consensus.cc:729] T 7dcc4cd7fd304dcfab6ac4f12b65f3b9 P 9f631ff72c7247ba8464150a716d2359 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f631ff72c7247ba8464150a716d2359, State: Initialized, Role: FOLLOWER
I20260806 22:51:04.953253 1538 raft_consensus.cc:348] T 7dcc4cd7fd304dcfab6ac4f12b65f3b9 P 9272551bb766498dbb8924f72a650d6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdbabcae759943ac8a6e7761e40de524" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 46143 } } peers { permanent_uuid: "9f631ff72c7247ba8464150a716d2359" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 37329 } } peers { permanent_uuid: "9272551bb766498dbb8924f72a650d6c" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 38043 } }
I20260806 22:51:04.953428 1538 raft_consensus.cc:374] T 7dcc4cd7fd304dcfab6ac4f12b65f3b9 P 9272551bb766498dbb8924f72a650d6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:51:04.953411 1540 consensus_queue.cc:260] T 7dcc4cd7fd304dcfab6ac4f12b65f3b9 P 9f631ff72c7247ba8464150a716d2359 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdbabcae759943ac8a6e7761e40de524" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 46143 } } peers { permanent_uuid: "9f631ff72c7247ba8464150a716d2359" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 37329 } } peers { permanent_uuid: "9272551bb766498dbb8924f72a650d6c" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 38043 } }
I20260806 22:51:04.953470 1538 raft_consensus.cc:729] T 7dcc4cd7fd304dcfab6ac4f12b65f3b9 P 9272551bb766498dbb8924f72a650d6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9272551bb766498dbb8924f72a650d6c, State: Initialized, Role: FOLLOWER
I20260806 22:51:04.953610 1538 consensus_queue.cc:260] T 7dcc4cd7fd304dcfab6ac4f12b65f3b9 P 9272551bb766498dbb8924f72a650d6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdbabcae759943ac8a6e7761e40de524" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 46143 } } peers { permanent_uuid: "9f631ff72c7247ba8464150a716d2359" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 37329 } } peers { permanent_uuid: "9272551bb766498dbb8924f72a650d6c" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 38043 } }
I20260806 22:51:04.953730 1540 ts_tablet_manager.cc:1428] T 7dcc4cd7fd304dcfab6ac4f12b65f3b9 P 9f631ff72c7247ba8464150a716d2359: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:51:04.953747 1328 heartbeater.cc:503] Master 127.6.40.126:44517 was elected leader, sending a full tablet report...
I20260806 22:51:04.953981 1132 heartbeater.cc:503] Master 127.6.40.126:44517 was elected leader, sending a full tablet report...
I20260806 22:51:04.953963 1538 ts_tablet_manager.cc:1428] T 7dcc4cd7fd304dcfab6ac4f12b65f3b9 P 9272551bb766498dbb8924f72a650d6c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:51:04.954377 1543 raft_consensus.cc:348] T 7dcc4cd7fd304dcfab6ac4f12b65f3b9 P bdbabcae759943ac8a6e7761e40de524 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdbabcae759943ac8a6e7761e40de524" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 46143 } } peers { permanent_uuid: "9f631ff72c7247ba8464150a716d2359" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 37329 } } peers { permanent_uuid: "9272551bb766498dbb8924f72a650d6c" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 38043 } }
I20260806 22:51:04.954545 1543 raft_consensus.cc:374] T 7dcc4cd7fd304dcfab6ac4f12b65f3b9 P bdbabcae759943ac8a6e7761e40de524 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:51:04.954581 1543 raft_consensus.cc:729] T 7dcc4cd7fd304dcfab6ac4f12b65f3b9 P bdbabcae759943ac8a6e7761e40de524 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdbabcae759943ac8a6e7761e40de524, State: Initialized, Role: FOLLOWER
I20260806 22:51:04.954712 1543 consensus_queue.cc:260] T 7dcc4cd7fd304dcfab6ac4f12b65f3b9 P bdbabcae759943ac8a6e7761e40de524 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdbabcae759943ac8a6e7761e40de524" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 46143 } } peers { permanent_uuid: "9f631ff72c7247ba8464150a716d2359" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 37329 } } peers { permanent_uuid: "9272551bb766498dbb8924f72a650d6c" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 38043 } }
I20260806 22:51:04.955015 1543 ts_tablet_manager.cc:1428] T 7dcc4cd7fd304dcfab6ac4f12b65f3b9 P bdbabcae759943ac8a6e7761e40de524: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:51:04.955024 1511 heartbeater.cc:503] Master 127.6.40.126:44517 was elected leader, sending a full tablet report...
W20260806 22:51:04.993036 1329 tablet.cc:2369] T 7dcc4cd7fd304dcfab6ac4f12b65f3b9 P 9f631ff72c7247ba8464150a716d2359: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:51:05.051136 1549 raft_consensus.cc:482] T 7dcc4cd7fd304dcfab6ac4f12b65f3b9 P 9272551bb766498dbb8924f72a650d6c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:51:05.051314 1549 raft_consensus.cc:504] T 7dcc4cd7fd304dcfab6ac4f12b65f3b9 P 9272551bb766498dbb8924f72a650d6c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdbabcae759943ac8a6e7761e40de524" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 46143 } } peers { permanent_uuid: "9f631ff72c7247ba8464150a716d2359" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 37329 } } peers { permanent_uuid: "9272551bb766498dbb8924f72a650d6c" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 38043 } }
I20260806 22:51:05.051777 1549 leader_election.cc:288] T 7dcc4cd7fd304dcfab6ac4f12b65f3b9 P 9272551bb766498dbb8924f72a650d6c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bdbabcae759943ac8a6e7761e40de524 (127.6.40.67:46143), 9f631ff72c7247ba8464150a716d2359 (127.6.40.66:37329)
I20260806 22:51:05.055923 1532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:05.052106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.65:39545 (local address 127.6.40.66:37329)
0806 22:51:05.052163 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:05.052165 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:05.052298 (+ 133us) server_negotiation.cc:407] Connection header received
0806 22:51:05.052328 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:05.052329 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:05.052349 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:05.052386 (+ 37us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:51:05.052555 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:05.053344 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:05.054933 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:05.055542 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:05.055611 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:05.055623 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:05.055810 (+ 187us) server_negotiation.cc:299] Negotiation successful
0806 22:51:05.055850 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 22:51:05.055923 1535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:05.052099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.65:57555 (local address 127.6.40.67:46143)
0806 22:51:05.052150 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:05.052153 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:05.052182 (+ 29us) server_negotiation.cc:407] Connection header received
0806 22:51:05.052213 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:05.052214 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:05.052227 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:05.052283 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:51:05.052552 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:05.053344 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:05.054933 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:05.055535 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:05.055611 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:05.055623 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:05.055810 (+ 187us) server_negotiation.cc:299] Negotiation successful
0806 22:51:05.055850 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 22:51:05.055940 1134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:05.051969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.67:46143 (local address 127.6.40.65:57555)
0806 22:51:05.052038 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0806 22:51:05.052052 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 22:51:05.052128 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:51:05.052309 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:51:05.052310 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:51:05.052313 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:51:05.052441 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:05.052444 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:05.053437 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:05.053437 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:05.054825 (+ 1388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:05.054829 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:05.055627 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:05.055628 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:05.055724 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:05.055731 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 22:51:05.055791 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 22:51:05.055842 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":115}
I20260806 22:51:05.055943 1552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:05.051969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.66:37329 (local address 127.6.40.65:39545)
0806 22:51:05.052183 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0806 22:51:05.052193 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:51:05.052249 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:51:05.052416 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:51:05.052417 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:51:05.052419 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:51:05.052485 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:05.052486 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:05.053437 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:05.053437 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:05.054825 (+ 1388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:05.054829 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:05.055627 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:05.055628 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:05.055724 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:05.055731 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 22:51:05.055791 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 22:51:05.055842 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"mutex_wait_us":135,"thread_start_us":71,"threads_started":1}
I20260806 22:51:05.056439 1452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7dcc4cd7fd304dcfab6ac4f12b65f3b9" candidate_uuid: "9272551bb766498dbb8924f72a650d6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdbabcae759943ac8a6e7761e40de524" is_pre_election: true
I20260806 22:51:05.056636 1452 raft_consensus.cc:2454] T 7dcc4cd7fd304dcfab6ac4f12b65f3b9 P bdbabcae759943ac8a6e7761e40de524 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9272551bb766498dbb8924f72a650d6c in term 0.
I20260806 22:51:05.056644 1261 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7dcc4cd7fd304dcfab6ac4f12b65f3b9" candidate_uuid: "9272551bb766498dbb8924f72a650d6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f631ff72c7247ba8464150a716d2359" is_pre_election: true
I20260806 22:51:05.056807 1261 raft_consensus.cc:2454] T 7dcc4cd7fd304dcfab6ac4f12b65f3b9 P 9f631ff72c7247ba8464150a716d2359 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9272551bb766498dbb8924f72a650d6c in term 0.
I20260806 22:51:05.056895 987 leader_election.cc:302] T 7dcc4cd7fd304dcfab6ac4f12b65f3b9 P 9272551bb766498dbb8924f72a650d6c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9272551bb766498dbb8924f72a650d6c, bdbabcae759943ac8a6e7761e40de524; no voters:
I20260806 22:51:05.057044 1549 raft_consensus.cc:2781] T 7dcc4cd7fd304dcfab6ac4f12b65f3b9 P 9272551bb766498dbb8924f72a650d6c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:51:05.057094 1549 raft_consensus.cc:482] T 7dcc4cd7fd304dcfab6ac4f12b65f3b9 P 9272551bb766498dbb8924f72a650d6c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:51:05.057123 1549 raft_consensus.cc:3037] T 7dcc4cd7fd304dcfab6ac4f12b65f3b9 P 9272551bb766498dbb8924f72a650d6c [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:51:05.057893 1549 raft_consensus.cc:504] T 7dcc4cd7fd304dcfab6ac4f12b65f3b9 P 9272551bb766498dbb8924f72a650d6c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdbabcae759943ac8a6e7761e40de524" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 46143 } } peers { permanent_uuid: "9f631ff72c7247ba8464150a716d2359" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 37329 } } peers { permanent_uuid: "9272551bb766498dbb8924f72a650d6c" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 38043 } }
I20260806 22:51:05.058050 1549 leader_election.cc:288] T 7dcc4cd7fd304dcfab6ac4f12b65f3b9 P 9272551bb766498dbb8924f72a650d6c [CANDIDATE]: Term 1 election: Requested vote from peers bdbabcae759943ac8a6e7761e40de524 (127.6.40.67:46143), 9f631ff72c7247ba8464150a716d2359 (127.6.40.66:37329)
I20260806 22:51:05.058225 1452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7dcc4cd7fd304dcfab6ac4f12b65f3b9" candidate_uuid: "9272551bb766498dbb8924f72a650d6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdbabcae759943ac8a6e7761e40de524"
I20260806 22:51:05.058254 1261 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7dcc4cd7fd304dcfab6ac4f12b65f3b9" candidate_uuid: "9272551bb766498dbb8924f72a650d6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f631ff72c7247ba8464150a716d2359"
I20260806 22:51:05.058305 1452 raft_consensus.cc:3037] T 7dcc4cd7fd304dcfab6ac4f12b65f3b9 P bdbabcae759943ac8a6e7761e40de524 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:51:05.058357 1261 raft_consensus.cc:3037] T 7dcc4cd7fd304dcfab6ac4f12b65f3b9 P 9f631ff72c7247ba8464150a716d2359 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:51:05.059142 1452 raft_consensus.cc:2454] T 7dcc4cd7fd304dcfab6ac4f12b65f3b9 P bdbabcae759943ac8a6e7761e40de524 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9272551bb766498dbb8924f72a650d6c in term 1.
I20260806 22:51:05.059144 1261 raft_consensus.cc:2454] T 7dcc4cd7fd304dcfab6ac4f12b65f3b9 P 9f631ff72c7247ba8464150a716d2359 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9272551bb766498dbb8924f72a650d6c in term 1.
I20260806 22:51:05.059314 987 leader_election.cc:302] T 7dcc4cd7fd304dcfab6ac4f12b65f3b9 P 9272551bb766498dbb8924f72a650d6c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9272551bb766498dbb8924f72a650d6c, bdbabcae759943ac8a6e7761e40de524; no voters:
I20260806 22:51:05.059422 1549 raft_consensus.cc:2781] T 7dcc4cd7fd304dcfab6ac4f12b65f3b9 P 9272551bb766498dbb8924f72a650d6c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:51:05.059593 1549 raft_consensus.cc:686] T 7dcc4cd7fd304dcfab6ac4f12b65f3b9 P 9272551bb766498dbb8924f72a650d6c [term 1 LEADER]: Becoming Leader. State: Replica: 9272551bb766498dbb8924f72a650d6c, State: Running, Role: LEADER
I20260806 22:51:05.059719 1549 consensus_queue.cc:237] T 7dcc4cd7fd304dcfab6ac4f12b65f3b9 P 9272551bb766498dbb8924f72a650d6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdbabcae759943ac8a6e7761e40de524" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 46143 } } peers { permanent_uuid: "9f631ff72c7247ba8464150a716d2359" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 37329 } } peers { permanent_uuid: "9272551bb766498dbb8924f72a650d6c" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 38043 } }
I20260806 22:51:05.060712 874 catalog_manager.cc:5495] T 7dcc4cd7fd304dcfab6ac4f12b65f3b9 P 9272551bb766498dbb8924f72a650d6c reported cstate change: term changed from 0 to 1, leader changed from <none> to 9272551bb766498dbb8924f72a650d6c (127.6.40.65). New cstate: current_term: 1 leader_uuid: "9272551bb766498dbb8924f72a650d6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdbabcae759943ac8a6e7761e40de524" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 46143 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9f631ff72c7247ba8464150a716d2359" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 37329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9272551bb766498dbb8924f72a650d6c" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 38043 } health_report { overall_health: HEALTHY } } }
W20260806 22:51:05.070466 1133 tablet.cc:2369] T 7dcc4cd7fd304dcfab6ac4f12b65f3b9 P 9272551bb766498dbb8924f72a650d6c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:51:05.089986 1534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:05.087366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44184 (local address 127.6.40.65:38043)
0806 22:51:05.087435 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:05.087437 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:05.087446 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:51:05.087520 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:05.087521 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:05.087545 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:05.087633 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:51:05.087788 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:05.088319 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:05.089150 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:05.089380 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:05.089459 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:05.089638 (+ 179us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:51:05.089752 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:51:05.089848 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:05.089868 (+ 20us) server_negotiation.cc:299] Negotiation successful
0806 22:51:05.089886 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 22:51:05.091835 1452 raft_consensus.cc:1261] T 7dcc4cd7fd304dcfab6ac4f12b65f3b9 P bdbabcae759943ac8a6e7761e40de524 [term 1 FOLLOWER]: Refusing update from remote peer 9272551bb766498dbb8924f72a650d6c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:51:05.091866 1261 raft_consensus.cc:1261] T 7dcc4cd7fd304dcfab6ac4f12b65f3b9 P 9f631ff72c7247ba8464150a716d2359 [term 1 FOLLOWER]: Refusing update from remote peer 9272551bb766498dbb8924f72a650d6c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:51:05.092221 1554 consensus_queue.cc:1035] T 7dcc4cd7fd304dcfab6ac4f12b65f3b9 P 9272551bb766498dbb8924f72a650d6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "bdbabcae759943ac8a6e7761e40de524" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 46143 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:51:05.092353 1549 consensus_queue.cc:1035] T 7dcc4cd7fd304dcfab6ac4f12b65f3b9 P 9272551bb766498dbb8924f72a650d6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f631ff72c7247ba8464150a716d2359" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 37329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:51:05.095345 1559 mvcc.cc:205] Tried to move back new op lower bound from 7315688100213411840 to 7315688100085276672. Current Snapshot: MvccSnapshot[applied={T|T < 7315688100213411840}]
I20260806 22:51:05.095911 1561 mvcc.cc:205] Tried to move back new op lower bound from 7315688100213411840 to 7315688100085276672. Current Snapshot: MvccSnapshot[applied={T|T < 7315688100213411840}]
I20260806 22:51:05.096028 1560 mvcc.cc:205] Tried to move back new op lower bound from 7315688100213411840 to 7315688100085276672. Current Snapshot: MvccSnapshot[applied={T|T < 7315688100213411840 or (T in {7315688100213411840})}]
I20260806 22:51:05.098361 872 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58574:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:51:05.098464 872 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58574:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:51:05.099253 872 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ae4c8b0f0aa141efab70e26d501c8c6f: Sending DeleteTablet for 3 replicas of tablet 7dcc4cd7fd304dcfab6ac4f12b65f3b9
I20260806 22:51:05.099611 1423 tablet_service.cc:1514] Processing DeleteTablet for tablet 7dcc4cd7fd304dcfab6ac4f12b65f3b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:51:05 UTC) from {username='kudu', principal='kudu/127.6.40.126@KRBTEST.COM'} at 127.0.0.1:37326
I20260806 22:51:05.099850 1570 tablet_replica.cc:330] T 7dcc4cd7fd304dcfab6ac4f12b65f3b9 P bdbabcae759943ac8a6e7761e40de524: stopping tablet replica
I20260806 22:51:05.099850 1234 tablet_service.cc:1514] Processing DeleteTablet for tablet 7dcc4cd7fd304dcfab6ac4f12b65f3b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:51:05 UTC) from {username='kudu', principal='kudu/127.6.40.126@KRBTEST.COM'} at 127.0.0.1:39426
I20260806 22:51:05.099972 1570 raft_consensus.cc:2229] T 7dcc4cd7fd304dcfab6ac4f12b65f3b9 P bdbabcae759943ac8a6e7761e40de524 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:51:05.100093 1570 raft_consensus.cc:2258] T 7dcc4cd7fd304dcfab6ac4f12b65f3b9 P bdbabcae759943ac8a6e7761e40de524 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:51:05.100111 1571 tablet_replica.cc:330] T 7dcc4cd7fd304dcfab6ac4f12b65f3b9 P 9f631ff72c7247ba8464150a716d2359: stopping tablet replica
I20260806 22:51:05.100168 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 956
I20260806 22:51:05.107618 1571 raft_consensus.cc:2229] T 7dcc4cd7fd304dcfab6ac4f12b65f3b9 P 9f631ff72c7247ba8464150a716d2359 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:51:05.107688 1570 ts_tablet_manager.cc:1905] T 7dcc4cd7fd304dcfab6ac4f12b65f3b9 P bdbabcae759943ac8a6e7761e40de524: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:51:05.107765 1571 raft_consensus.cc:2258] T 7dcc4cd7fd304dcfab6ac4f12b65f3b9 P 9f631ff72c7247ba8464150a716d2359 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:51:05.108240 1571 ts_tablet_manager.cc:1905] T 7dcc4cd7fd304dcfab6ac4f12b65f3b9 P 9f631ff72c7247ba8464150a716d2359: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:51:05.109575 1570 ts_tablet_manager.cc:1918] T 7dcc4cd7fd304dcfab6ac4f12b65f3b9 P bdbabcae759943ac8a6e7761e40de524: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:51:05.109653 1570 log.cc:1196] T 7dcc4cd7fd304dcfab6ac4f12b65f3b9 P bdbabcae759943ac8a6e7761e40de524: Deleting WAL directory at /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/minicluster-data/ts-2/wal/wals/7dcc4cd7fd304dcfab6ac4f12b65f3b9
I20260806 22:51:05.109956 1570 ts_tablet_manager.cc:1939] T 7dcc4cd7fd304dcfab6ac4f12b65f3b9 P bdbabcae759943ac8a6e7761e40de524: Deleting consensus metadata
I20260806 22:51:05.110062 1571 ts_tablet_manager.cc:1918] T 7dcc4cd7fd304dcfab6ac4f12b65f3b9 P 9f631ff72c7247ba8464150a716d2359: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:51:05.110138 1571 log.cc:1196] T 7dcc4cd7fd304dcfab6ac4f12b65f3b9 P 9f631ff72c7247ba8464150a716d2359: Deleting WAL directory at /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056405724609-6305-0/minicluster-data/ts-1/wal/wals/7dcc4cd7fd304dcfab6ac4f12b65f3b9
I20260806 22:51:05.110424 1571 ts_tablet_manager.cc:1939] T 7dcc4cd7fd304dcfab6ac4f12b65f3b9 P 9f631ff72c7247ba8464150a716d2359: Deleting consensus metadata
I20260806 22:51:05.110468 854 catalog_manager.cc:4841] TS bdbabcae759943ac8a6e7761e40de524 (127.6.40.67:46143): tablet 7dcc4cd7fd304dcfab6ac4f12b65f3b9 (table test-table [id=5310dc00b92a46deb24b75c65342e383]) successfully deleted
I20260806 22:51:05.110878 851 catalog_manager.cc:4841] TS 9f631ff72c7247ba8464150a716d2359 (127.6.40.66:37329): tablet 7dcc4cd7fd304dcfab6ac4f12b65f3b9 (table test-table [id=5310dc00b92a46deb24b75c65342e383]) successfully deleted
W20260806 22:51:05.112075 851 connection.cc:535] client connection to 127.6.40.65:38043 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260806 22:51:05.112180 851 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 22:51:05.112725 1533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:05.112569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.65:38043 (local address 127.0.0.1:44194)
0806 22:51:05.112630 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0806 22:51:05.112685 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.65:38043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260806 22:51:05.112776 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 1139
W20260806 22:51:05.112896 851 catalog_manager.cc:4584] TS 9272551bb766498dbb8924f72a650d6c (127.6.40.65:38043): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7dcc4cd7fd304dcfab6ac4f12b65f3b9: Network error: Client connection negotiation failed: client connection to 127.6.40.65:38043: connect: Connection refused (error 111)
I20260806 22:51:05.124943 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 1335
I20260806 22:51:05.134649 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 826
2026-08-06T22:51:05Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2255 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-06T22:51:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:51:05Z Disabled control of system clock
I20260806 22:51:05.169498 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056405724609-6305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056405724609-6305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056405724609-6305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.126:34025
--webserver_interface=127.6.40.126
--webserver_port=0
--builtin_ntp_servers=127.6.40.84:38991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.40.126:34025
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 22:51:05.251986 1576 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:51:05.252242 1576 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:51:05.252279 1576 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:51:05.253440 1576 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:51:05.253520 1576 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:51:05.253545 1576 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:51:05.253573 1576 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:51:05.253602 1576 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:51:05.254921 1576 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:38991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056405724609-6305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056405724609-6305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.40.126:34025
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.126:34025
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1576
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056405724609-6305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:51:05.255192 1576 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:51:05.255488 1576 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:51:05.260319 1582 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:51:05.260336 1585 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:51:05.260466 1576 server_base.cc:1034] running on GCE node
W20260806 22:51:05.260336 1583 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:51:05.260918 1576 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:51:05.261281 1576 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:51:05.262444 1576 hybrid_clock.cc:648] HybridClock initialized: now 1786056665262424 us; error 42 us; skew 500 ppm
I20260806 22:51:05.263746 1576 webserver.cc:460] Webserver started at http://127.6.40.126:42245/ using document root <none> and password file <none>
I20260806 22:51:05.263989 1576 fs_manager.cc:359] Metadata directory not provided
I20260806 22:51:05.264068 1576 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:51:05.264228 1576 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:51:05.265324 1576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056405724609-6305-0/minicluster-data/master-0/data/instance:
uuid: "721ea693df0846a5bfbece38ac1aa18f"
format_stamp: "Formatted at 2026-08-06 22:51:05 on dist-test-slave-zh2d"
server_key: "68f8473a589f5146f141424567b522f2"
server_key_iv: "8bc7ee6863a36bca7746e1c3d7c6ea2d"
server_key_version: "encryptionkey@0"
I20260806 22:51:05.265693 1576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056405724609-6305-0/minicluster-data/master-0/wal/instance:
uuid: "721ea693df0846a5bfbece38ac1aa18f"
format_stamp: "Formatted at 2026-08-06 22:51:05 on dist-test-slave-zh2d"
server_key: "68f8473a589f5146f141424567b522f2"
server_key_iv: "8bc7ee6863a36bca7746e1c3d7c6ea2d"
server_key_version: "encryptionkey@0"
I20260806 22:51:05.266988 1576 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:51:05.267742 1592 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:51:05.267931 1576 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:51:05.268042 1576 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056405724609-6305-0/minicluster-data/master-0/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056405724609-6305-0/minicluster-data/master-0/wal
uuid: "721ea693df0846a5bfbece38ac1aa18f"
format_stamp: "Formatted at 2026-08-06 22:51:05 on dist-test-slave-zh2d"
server_key: "68f8473a589f5146f141424567b522f2"
server_key_iv: "8bc7ee6863a36bca7746e1c3d7c6ea2d"
server_key_version: "encryptionkey@0"
I20260806 22:51:05.268115 1576 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056405724609-6305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056405724609-6305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056405724609-6305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:51:05.285290 1576 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:51:05.285627 1576 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:51:05.285789 1576 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:51:05.290194 1576 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.126:34025
I20260806 22:51:05.290248 1668 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.126:34025 every 8 connection(s)
I20260806 22:51:05.290596 1576 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
I20260806 22:51:05.291630 1669 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:51:05.293905 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 1576
I20260806 22:51:05.293994 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056405724609-6305-0/minicluster-data/master-0/wal/instance
I20260806 22:51:05.294149 1669 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 721ea693df0846a5bfbece38ac1aa18f: Bootstrap starting.
I20260806 22:51:05.294216 6305 external_mini_cluster.cc:1428] Setting key 42d26d1072b57b6cdb6b686f4d9f08d8
I20260806 22:51:05.294957 1669 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 721ea693df0846a5bfbece38ac1aa18f: Neither blocks nor log segments found. Creating new log.
I20260806 22:51:05.295300 1669 log.cc:824] T 00000000000000000000000000000000 P 721ea693df0846a5bfbece38ac1aa18f: Log is configured to *not* fsync() on all Append() calls
I20260806 22:51:05.296067 1669 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 721ea693df0846a5bfbece38ac1aa18f: No bootstrap required, opened a new log
I20260806 22:51:05.298446 1669 raft_consensus.cc:348] T 00000000000000000000000000000000 P 721ea693df0846a5bfbece38ac1aa18f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "721ea693df0846a5bfbece38ac1aa18f" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 34025 } }
I20260806 22:51:05.298612 1669 raft_consensus.cc:374] T 00000000000000000000000000000000 P 721ea693df0846a5bfbece38ac1aa18f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:51:05.298645 1669 raft_consensus.cc:729] T 00000000000000000000000000000000 P 721ea693df0846a5bfbece38ac1aa18f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 721ea693df0846a5bfbece38ac1aa18f, State: Initialized, Role: FOLLOWER
I20260806 22:51:05.298763 1669 consensus_queue.cc:260] T 00000000000000000000000000000000 P 721ea693df0846a5bfbece38ac1aa18f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "721ea693df0846a5bfbece38ac1aa18f" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 34025 } }
I20260806 22:51:05.298833 1669 raft_consensus.cc:388] T 00000000000000000000000000000000 P 721ea693df0846a5bfbece38ac1aa18f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:51:05.298866 1669 raft_consensus.cc:482] T 00000000000000000000000000000000 P 721ea693df0846a5bfbece38ac1aa18f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:51:05.298918 1669 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 721ea693df0846a5bfbece38ac1aa18f [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:51:05.299732 1669 raft_consensus.cc:504] T 00000000000000000000000000000000 P 721ea693df0846a5bfbece38ac1aa18f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "721ea693df0846a5bfbece38ac1aa18f" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 34025 } }
I20260806 22:51:05.299846 1669 leader_election.cc:302] T 00000000000000000000000000000000 P 721ea693df0846a5bfbece38ac1aa18f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 721ea693df0846a5bfbece38ac1aa18f; no voters:
I20260806 22:51:05.300031 1669 leader_election.cc:288] T 00000000000000000000000000000000 P 721ea693df0846a5bfbece38ac1aa18f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:51:05.300118 1676 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 721ea693df0846a5bfbece38ac1aa18f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:51:05.300278 1676 raft_consensus.cc:686] T 00000000000000000000000000000000 P 721ea693df0846a5bfbece38ac1aa18f [term 1 LEADER]: Becoming Leader. State: Replica: 721ea693df0846a5bfbece38ac1aa18f, State: Running, Role: LEADER
I20260806 22:51:05.300390 1676 consensus_queue.cc:237] T 00000000000000000000000000000000 P 721ea693df0846a5bfbece38ac1aa18f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "721ea693df0846a5bfbece38ac1aa18f" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 34025 } }
I20260806 22:51:05.300670 1674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:05.294778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34036 (local address 127.6.40.126:34025)
0806 22:51:05.295016 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:05.295025 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:05.295061 (+ 36us) server_negotiation.cc:407] Connection header received
0806 22:51:05.295650 (+ 589us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:05.295657 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:05.295739 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:05.295896 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:05.296091 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:05.297514 (+ 1423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:05.298285 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:05.298907 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:05.299033 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:05.299744 (+ 711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:51:05.299749 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:51:05.299749 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:51:05.299976 (+ 227us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:51:05.300025 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:51:05.300041 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:51:05.300111 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:05.300314 (+ 203us) server_negotiation.cc:299] Negotiation successful
0806 22:51:05.300404 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":91,"threads_started":1}
I20260806 22:51:05.300395 1669 sys_catalog.cc:564] T 00000000000000000000000000000000 P 721ea693df0846a5bfbece38ac1aa18f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:51:05.300915 1679 sys_catalog.cc:455] T 00000000000000000000000000000000 P 721ea693df0846a5bfbece38ac1aa18f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 721ea693df0846a5bfbece38ac1aa18f. Latest consensus state: current_term: 1 leader_uuid: "721ea693df0846a5bfbece38ac1aa18f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "721ea693df0846a5bfbece38ac1aa18f" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 34025 } } }
I20260806 22:51:05.300930 1677 sys_catalog.cc:455] T 00000000000000000000000000000000 P 721ea693df0846a5bfbece38ac1aa18f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "721ea693df0846a5bfbece38ac1aa18f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "721ea693df0846a5bfbece38ac1aa18f" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 34025 } } }
I20260806 22:51:05.301026 1677 sys_catalog.cc:458] T 00000000000000000000000000000000 P 721ea693df0846a5bfbece38ac1aa18f [sys.catalog]: This master's current role is: LEADER
I20260806 22:51:05.301021 1679 sys_catalog.cc:458] T 00000000000000000000000000000000 P 721ea693df0846a5bfbece38ac1aa18f [sys.catalog]: This master's current role is: LEADER
I20260806 22:51:05.301388 1681 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:51:05.301891 1681 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:51:05.303469 1681 catalog_manager.cc:1347] Generated new cluster ID: 09c77e875366499eaedda89a62601fa7
I20260806 22:51:05.303530 1681 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:51:05.329485 1681 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:51:05.330072 1681 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:51:05.335420 1681 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 721ea693df0846a5bfbece38ac1aa18f: Generated new TSK 0
I20260806 22:51:05.335621 1681 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:51:05.344259 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056405724609-6305-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056405724609-6305-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056405724609-6305-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.65:0
--local_ip_for_outbound_sockets=127.6.40.65
--webserver_interface=127.6.40.65
--webserver_port=0
--tserver_master_addrs=127.6.40.126:34025
--builtin_ntp_servers=127.6.40.84:38991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 22:51:05.442544 1698 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:51:05.442776 1698 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:51:05.442816 1698 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:51:05.443964 1698 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:51:05.444028 1698 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:51:05.444069 1698 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.65
I20260806 22:51:05.445508 1698 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:38991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056405724609-6305-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056405724609-6305-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.40.65
--webserver_port=0
--tserver_master_addrs=127.6.40.126:34025
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1698
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.65
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056405724609-6305-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:51:05.445782 1698 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:51:05.446116 1698 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:51:05.451442 1705 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:51:05.451515 1708 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:51:05.451495 1706 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:51:05.451798 1698 server_base.cc:1034] running on GCE node
I20260806 22:51:05.452034 1698 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:51:05.452319 1698 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:51:05.453490 1698 hybrid_clock.cc:648] HybridClock initialized: now 1786056665453477 us; error 33 us; skew 500 ppm
I20260806 22:51:05.454725 1698 webserver.cc:460] Webserver started at http://127.6.40.65:45887/ using document root <none> and password file <none>
I20260806 22:51:05.455001 1698 fs_manager.cc:359] Metadata directory not provided
I20260806 22:51:05.455076 1698 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:51:05.455235 1698 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:51:05.456313 1698 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056405724609-6305-0/minicluster-data/ts-0/data/instance:
uuid: "c5d439a6aafd41cdb6e2dc34afe07409"
format_stamp: "Formatted at 2026-08-06 22:51:05 on dist-test-slave-zh2d"
server_key: "e6dad57c7c48f1d64476d395417a24fa"
server_key_iv: "e055608c7fda701624fbe3616bf1d8e8"
server_key_version: "encryptionkey@0"
I20260806 22:51:05.456700 1698 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056405724609-6305-0/minicluster-data/ts-0/wal/instance:
uuid: "c5d439a6aafd41cdb6e2dc34afe07409"
format_stamp: "Formatted at 2026-08-06 22:51:05 on dist-test-slave-zh2d"
server_key: "e6dad57c7c48f1d64476d395417a24fa"
server_key_iv: "e055608c7fda701624fbe3616bf1d8e8"
server_key_version: "encryptionkey@0"
I20260806 22:51:05.458091 1698 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:51:05.458902 1715 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:51:05.459105 1698 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:51:05.459205 1698 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056405724609-6305-0/minicluster-data/ts-0/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056405724609-6305-0/minicluster-data/ts-0/wal
uuid: "c5d439a6aafd41cdb6e2dc34afe07409"
format_stamp: "Formatted at 2026-08-06 22:51:05 on dist-test-slave-zh2d"
server_key: "e6dad57c7c48f1d64476d395417a24fa"
server_key_iv: "e055608c7fda701624fbe3616bf1d8e8"
server_key_version: "encryptionkey@0"
I20260806 22:51:05.459288 1698 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056405724609-6305-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056405724609-6305-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056405724609-6305-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:51:05.497706 1698 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:51:05.498076 1698 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:51:05.498236 1698 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:51:05.498492 1698 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:51:05.498857 1698 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:51:05.498914 1698 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:51:05.498956 1698 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:51:05.498983 1698 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:51:05.505342 1698 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.65:40159
I20260806 22:51:05.505438 1867 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.65:40159 every 8 connection(s)
I20260806 22:51:05.505774 1698 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056405724609-6305-0/minicluster-data/ts-0/data/info.pb
I20260806 22:51:05.511356 1872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:05.506553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:34025 (local address 127.6.40.65:57671)
0806 22:51:05.506756 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0806 22:51:05.506792 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 22:51:05.507286 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:51:05.507537 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:51:05.507540 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:51:05.507549 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:51:05.508071 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:05.508076 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:05.508826 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:05.508827 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:05.509730 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:05.509733 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:05.510117 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:05.510117 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:05.510240 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:05.510804 (+ 564us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:51:05.510805 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:51:05.510835 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:51:05.510857 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:51:05.511037 (+ 180us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:51:05.511042 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:51:05.511044 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 22:51:05.511099 (+ 55us) client_negotiation.cc:240] Negotiation successful
0806 22:51:05.511186 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":92,"threads_started":1}
I20260806 22:51:05.511409 1674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:05.506561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.65:57671 (local address 127.6.40.126:34025)
0806 22:51:05.506634 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:05.506636 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:05.506889 (+ 253us) server_negotiation.cc:407] Connection header received
0806 22:51:05.507376 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:05.507378 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:05.507392 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:05.507450 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:05.508187 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:05.508726 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:05.509822 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:05.510026 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:05.510089 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:05.510946 (+ 857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:51:05.510947 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:51:05.510947 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:51:05.510952 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:51:05.510967 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:51:05.510971 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:51:05.511021 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:05.511221 (+ 200us) server_negotiation.cc:299] Negotiation successful
0806 22:51:05.511267 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 22:51:05.511415 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 1698
I20260806 22:51:05.511511 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056405724609-6305-0/minicluster-data/ts-0/wal/instance
I20260806 22:51:05.511720 6305 external_mini_cluster.cc:1428] Setting key ccf0ff565662dbfc6e5cf9bf6b500ed0
I20260806 22:51:05.512365 1868 heartbeater.cc:347] Connected to a master server at 127.6.40.126:34025
I20260806 22:51:05.512483 1868 heartbeater.cc:464] Registering TS with master...
I20260806 22:51:05.512691 1868 heartbeater.cc:511] Master 127.6.40.126:34025 requested a full tablet report, sending...
I20260806 22:51:05.512787 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056405724609-6305-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056405724609-6305-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056405724609-6305-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.66:0
--local_ip_for_outbound_sockets=127.6.40.66
--webserver_interface=127.6.40.66
--webserver_port=0
--tserver_master_addrs=127.6.40.126:34025
--builtin_ntp_servers=127.6.40.84:38991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 22:51:05.513314 1613 ts_manager.cc:194] Registered new tserver with Master: c5d439a6aafd41cdb6e2dc34afe07409 (127.6.40.65:40159)
I20260806 22:51:05.514223 1613 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.40.65:57671
W20260806 22:51:05.598083 1874 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:51:05.598305 1874 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:51:05.598328 1874 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:51:05.599448 1874 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:51:05.599483 1874 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:51:05.599505 1874 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.66
I20260806 22:51:05.600929 1874 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:38991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056405724609-6305-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056405724609-6305-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.40.66
--webserver_port=0
--tserver_master_addrs=127.6.40.126:34025
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1874
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.66
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056405724609-6305-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:51:05.601117 1874 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:51:05.601372 1874 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:51:05.606611 1889 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:51:05.606688 1887 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:51:05.606817 1874 server_base.cc:1034] running on GCE node
W20260806 22:51:05.606671 1883 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:51:05.607100 1874 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:51:05.607388 1874 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:51:05.608552 1874 hybrid_clock.cc:648] HybridClock initialized: now 1786056665608523 us; error 49 us; skew 500 ppm
I20260806 22:51:05.609835 1874 webserver.cc:460] Webserver started at http://127.6.40.66:37507/ using document root <none> and password file <none>
I20260806 22:51:05.610082 1874 fs_manager.cc:359] Metadata directory not provided
I20260806 22:51:05.610154 1874 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:51:05.610311 1874 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:51:05.611370 1874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056405724609-6305-0/minicluster-data/ts-1/data/instance:
uuid: "6b923d20c1c2436d9fa8761461ef9e50"
format_stamp: "Formatted at 2026-08-06 22:51:05 on dist-test-slave-zh2d"
server_key: "6ebb9d1add4b6c990e285a4b71a2155d"
server_key_iv: "8b2539c342c0bbbc5c1211f00ef892b8"
server_key_version: "encryptionkey@0"
I20260806 22:51:05.611744 1874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056405724609-6305-0/minicluster-data/ts-1/wal/instance:
uuid: "6b923d20c1c2436d9fa8761461ef9e50"
format_stamp: "Formatted at 2026-08-06 22:51:05 on dist-test-slave-zh2d"
server_key: "6ebb9d1add4b6c990e285a4b71a2155d"
server_key_iv: "8b2539c342c0bbbc5c1211f00ef892b8"
server_key_version: "encryptionkey@0"
I20260806 22:51:05.613150 1874 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:51:05.613947 1897 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:51:05.614228 1874 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:51:05.614331 1874 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056405724609-6305-0/minicluster-data/ts-1/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056405724609-6305-0/minicluster-data/ts-1/wal
uuid: "6b923d20c1c2436d9fa8761461ef9e50"
format_stamp: "Formatted at 2026-08-06 22:51:05 on dist-test-slave-zh2d"
server_key: "6ebb9d1add4b6c990e285a4b71a2155d"
server_key_iv: "8b2539c342c0bbbc5c1211f00ef892b8"
server_key_version: "encryptionkey@0"
I20260806 22:51:05.614404 1874 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056405724609-6305-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056405724609-6305-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056405724609-6305-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:51:05.652447 1874 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:51:05.652755 1874 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:51:05.652881 1874 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:51:05.653100 1874 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:51:05.653486 1874 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:51:05.653522 1874 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:51:05.653579 1874 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:51:05.653605 1874 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:51:05.659654 1874 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.66:46547
I20260806 22:51:05.659755 2056 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.66:46547 every 8 connection(s)
I20260806 22:51:05.660122 1874 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056405724609-6305-0/minicluster-data/ts-1/data/info.pb
I20260806 22:51:05.665882 1674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:05.661031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.66:42351 (local address 127.6.40.126:34025)
0806 22:51:05.661090 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:05.661093 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:05.661391 (+ 298us) server_negotiation.cc:407] Connection header received
0806 22:51:05.661994 (+ 603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:05.661995 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:05.662010 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:05.662061 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:05.662767 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:05.663349 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:05.664387 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:05.664586 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:05.664660 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:05.665566 (+ 906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:51:05.665567 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:51:05.665568 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:51:05.665573 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:51:05.665591 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:51:05.665595 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:51:05.665655 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:05.665772 (+ 117us) server_negotiation.cc:299] Negotiation successful
0806 22:51:05.665800 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 22:51:05.666114 2062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:05.661045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:34025 (local address 127.6.40.66:42351)
0806 22:51:05.661251 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0806 22:51:05.661284 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 22:51:05.661862 (+ 578us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:51:05.662140 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:51:05.662143 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:51:05.662152 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:51:05.662671 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:05.662675 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:05.663458 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:05.663460 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:05.664284 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:05.664287 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:05.664670 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:05.664671 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:05.664803 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:05.665381 (+ 578us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:51:05.665382 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:51:05.665411 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:51:05.665435 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:51:05.665658 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:51:05.665665 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:51:05.665669 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:51:05.665749 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 22:51:05.665880 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":94,"threads_started":1}
I20260806 22:51:05.666625 2057 heartbeater.cc:347] Connected to a master server at 127.6.40.126:34025
I20260806 22:51:05.666705 2057 heartbeater.cc:464] Registering TS with master...
I20260806 22:51:05.666962 2057 heartbeater.cc:511] Master 127.6.40.126:34025 requested a full tablet report, sending...
I20260806 22:51:05.667394 1613 ts_manager.cc:194] Registered new tserver with Master: 6b923d20c1c2436d9fa8761461ef9e50 (127.6.40.66:46547)
I20260806 22:51:05.667970 1613 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.40.66:42351
I20260806 22:51:05.669067 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 1874
I20260806 22:51:05.669184 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056405724609-6305-0/minicluster-data/ts-1/wal/instance
I20260806 22:51:05.669399 6305 external_mini_cluster.cc:1428] Setting key 4491b730f76146b3240270615b883f77
I20260806 22:51:05.670608 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056405724609-6305-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056405724609-6305-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056405724609-6305-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.40.67:0
--local_ip_for_outbound_sockets=127.6.40.67
--webserver_interface=127.6.40.67
--webserver_port=0
--tserver_master_addrs=127.6.40.126:34025
--builtin_ntp_servers=127.6.40.84:38991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 22:51:05.754220 2063 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:51:05.754418 2063 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:51:05.754439 2063 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:51:05.755575 2063 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:51:05.755616 2063 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:51:05.755640 2063 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.40.67
I20260806 22:51:05.757098 2063 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:38991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056405724609-6305-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056405724609-6305-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.40.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.40.67
--webserver_port=0
--tserver_master_addrs=127.6.40.126:34025
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2063
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.40.67
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056405724609-6305-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:51:05.757334 2063 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:51:05.757663 2063 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:51:05.762933 2074 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:51:05.763006 2070 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:51:05.762995 2063 server_base.cc:1034] running on GCE node
W20260806 22:51:05.762949 2068 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:51:05.763451 2063 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:51:05.763769 2063 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:51:05.764926 2063 hybrid_clock.cc:648] HybridClock initialized: now 1786056665764902 us; error 36 us; skew 500 ppm
I20260806 22:51:05.766321 2063 webserver.cc:460] Webserver started at http://127.6.40.67:44161/ using document root <none> and password file <none>
I20260806 22:51:05.766669 2063 fs_manager.cc:359] Metadata directory not provided
I20260806 22:51:05.766757 2063 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:51:05.766944 2063 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:51:05.768240 2063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056405724609-6305-0/minicluster-data/ts-2/data/instance:
uuid: "e8a91b570d0d4a26bbfaebfacd50335b"
format_stamp: "Formatted at 2026-08-06 22:51:05 on dist-test-slave-zh2d"
server_key: "188f569974c9d547405491015f79f7af"
server_key_iv: "96101fe3806701008728e7b3b48ef177"
server_key_version: "encryptionkey@0"
I20260806 22:51:05.768689 2063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056405724609-6305-0/minicluster-data/ts-2/wal/instance:
uuid: "e8a91b570d0d4a26bbfaebfacd50335b"
format_stamp: "Formatted at 2026-08-06 22:51:05 on dist-test-slave-zh2d"
server_key: "188f569974c9d547405491015f79f7af"
server_key_iv: "96101fe3806701008728e7b3b48ef177"
server_key_version: "encryptionkey@0"
I20260806 22:51:05.770239 2063 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:51:05.771207 2083 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:51:05.771418 2063 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:51:05.771494 2063 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056405724609-6305-0/minicluster-data/ts-2/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056405724609-6305-0/minicluster-data/ts-2/wal
uuid: "e8a91b570d0d4a26bbfaebfacd50335b"
format_stamp: "Formatted at 2026-08-06 22:51:05 on dist-test-slave-zh2d"
server_key: "188f569974c9d547405491015f79f7af"
server_key_iv: "96101fe3806701008728e7b3b48ef177"
server_key_version: "encryptionkey@0"
I20260806 22:51:05.771575 2063 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056405724609-6305-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056405724609-6305-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056405724609-6305-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:51:05.780766 2063 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:51:05.781153 2063 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:51:05.781296 2063 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:51:05.781525 2063 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:51:05.781838 2063 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:51:05.781872 2063 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:51:05.781891 2063 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:51:05.781980 2063 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:51:05.787854 2063 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.67:43311
I20260806 22:51:05.787931 2235 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.67:43311 every 8 connection(s)
I20260806 22:51:05.788265 2063 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056405724609-6305-0/minicluster-data/ts-2/data/info.pb
I20260806 22:51:05.793368 1674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:05.789034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.67:48817 (local address 127.6.40.126:34025)
0806 22:51:05.789111 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:05.789114 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:05.789423 (+ 309us) server_negotiation.cc:407] Connection header received
0806 22:51:05.789886 (+ 463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:05.789887 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:05.789897 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:05.789950 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:05.790593 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:05.791083 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:05.792023 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:05.792226 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:05.792291 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:05.793157 (+ 866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:51:05.793159 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:51:05.793159 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:51:05.793163 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:51:05.793178 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:51:05.793182 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:51:05.793234 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:05.793284 (+ 50us) server_negotiation.cc:299] Negotiation successful
0806 22:51:05.793307 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 22:51:05.793553 2239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:05.789014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:34025 (local address 127.6.40.67:48817)
0806 22:51:05.789209 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0806 22:51:05.789242 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 22:51:05.789793 (+ 551us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:51:05.790000 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:51:05.790004 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:51:05.790017 (+ 13us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:51:05.790500 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:05.790504 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:05.791163 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:05.791163 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:05.791920 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:05.791922 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:05.792310 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:05.792311 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:05.792416 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:05.793015 (+ 599us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:51:05.793015 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:51:05.793053 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:51:05.793076 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:51:05.793237 (+ 161us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:51:05.793239 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:51:05.793241 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 22:51:05.793286 (+ 45us) client_negotiation.cc:240] Negotiation successful
0806 22:51:05.793358 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":109,"threads_started":1}
I20260806 22:51:05.794050 2236 heartbeater.cc:347] Connected to a master server at 127.6.40.126:34025
I20260806 22:51:05.794128 2236 heartbeater.cc:464] Registering TS with master...
I20260806 22:51:05.794327 2236 heartbeater.cc:511] Master 127.6.40.126:34025 requested a full tablet report, sending...
I20260806 22:51:05.794700 1613 ts_manager.cc:194] Registered new tserver with Master: e8a91b570d0d4a26bbfaebfacd50335b (127.6.40.67:43311)
I20260806 22:51:05.795029 1613 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.40.67:48817
I20260806 22:51:05.797102 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 2063
I20260806 22:51:05.797201 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056405724609-6305-0/minicluster-data/ts-2/wal/instance
I20260806 22:51:05.797427 6305 external_mini_cluster.cc:1428] Setting key 32a57cb35ee3ff6d6a7ebb2b7553dd85
I20260806 22:51:05.798553 6305 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:51:05.803268 1674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:05.800088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34094 (local address 127.6.40.126:34025)
0806 22:51:05.800128 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:05.800130 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:05.800239 (+ 109us) server_negotiation.cc:407] Connection header received
0806 22:51:05.800308 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:05.800309 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:05.800317 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:05.800401 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:05.800654 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:05.801264 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:05.802084 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:05.802278 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:05.802335 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:05.803075 (+ 740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:51:05.803076 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:51:05.803076 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:51:05.803080 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:51:05.803093 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:51:05.803097 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:51:05.803143 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:05.803197 (+ 54us) server_negotiation.cc:299] Negotiation successful
0806 22:51:05.803216 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 22:51:05.804878 1613 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34094:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:51:05.806493 1613 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:51:05.812798 2250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:05.808823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.66:46547 (local address 127.0.0.1:40820)
0806 22:51:05.809105 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0806 22:51:05.809115 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:51:05.809152 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:51:05.809481 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:51:05.809481 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:51:05.809483 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:51:05.809557 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:05.809559 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:05.810552 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:05.810554 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:05.811763 (+ 1209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:05.811765 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:05.812514 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:05.812515 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:05.812588 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:05.812601 (+ 13us) client_negotiation.cc:769] Sending connection context
0806 22:51:05.812675 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 22:51:05.812726 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"mutex_wait_us":2,"thread_start_us":72,"threads_started":1}
I20260806 22:51:05.812810 2256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:05.809048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40820 (local address 127.6.40.66:46547)
0806 22:51:05.809216 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:05.809220 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:05.809230 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:51:05.809275 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:05.809277 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:05.809288 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:05.809405 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:51:05.809649 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:05.810460 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:05.811860 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:05.812418 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:05.812508 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:05.812603 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:05.812690 (+ 87us) server_negotiation.cc:299] Negotiation successful
0806 22:51:05.812729 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":99,"threads_started":1}
I20260806 22:51:05.813123 2247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:05.808637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.65:40159 (local address 127.0.0.1:49554)
0806 22:51:05.808888 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0806 22:51:05.808930 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 22:51:05.809001 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:51:05.809341 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:51:05.809344 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:51:05.809346 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:51:05.809411 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:05.809413 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:05.810749 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:05.810750 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:05.811721 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:05.811723 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:05.812779 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:05.812780 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:05.812916 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:05.812922 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 22:51:05.812992 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 22:51:05.813035 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":58,"threads_started":1}
I20260806 22:51:05.813164 2251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:05.808746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49554 (local address 127.6.40.65:40159)
0806 22:51:05.809166 (+ 420us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:05.809171 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:05.809181 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:51:05.809221 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:05.809223 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:05.809238 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:05.809337 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:51:05.809507 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:05.810652 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:05.811828 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:05.812575 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:05.812650 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:05.812739 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:05.813025 (+ 286us) server_negotiation.cc:299] Negotiation successful
0806 22:51:05.813084 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":140,"threads_started":1}
I20260806 22:51:05.813395 2252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:05.808637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.67:43311 (local address 127.0.0.1:60152)
0806 22:51:05.808983 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0806 22:51:05.808993 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:51:05.809036 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:51:05.809663 (+ 627us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:51:05.809664 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:51:05.809666 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:51:05.809703 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:05.809704 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:05.810891 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:05.810892 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:05.811926 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:05.811928 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:05.813027 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:05.813028 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:05.813099 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:05.813103 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:51:05.813146 (+ 43us) client_negotiation.cc:240] Negotiation successful
0806 22:51:05.813329 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"mutex_wait_us":3,"thread_start_us":53,"threads_started":1}
I20260806 22:51:05.813935 1780 tablet_service.cc:1467] Processing CreateTablet for tablet 8c53cdccd609434d84bd5c1ba5d772ba (DEFAULT_TABLE table=test-table [id=2d0c29ccc40e40f8a291ec138c71f63f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:51:05.813930 1963 tablet_service.cc:1467] Processing CreateTablet for tablet 8c53cdccd609434d84bd5c1ba5d772ba (DEFAULT_TABLE table=test-table [id=2d0c29ccc40e40f8a291ec138c71f63f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:51:05.814289 1780 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c53cdccd609434d84bd5c1ba5d772ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:51:05.814296 1963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c53cdccd609434d84bd5c1ba5d772ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:51:05.817144 2258 tablet_bootstrap.cc:492] T 8c53cdccd609434d84bd5c1ba5d772ba P 6b923d20c1c2436d9fa8761461ef9e50: Bootstrap starting.
I20260806 22:51:05.817379 2257 tablet_bootstrap.cc:492] T 8c53cdccd609434d84bd5c1ba5d772ba P c5d439a6aafd41cdb6e2dc34afe07409: Bootstrap starting.
I20260806 22:51:05.817462 2254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:05.808856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60152 (local address 127.6.40.67:43311)
0806 22:51:05.809510 (+ 654us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:05.809515 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:05.809523 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:51:05.809566 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:05.809568 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:05.809582 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:05.809653 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:51:05.809757 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:05.810809 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:05.812007 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:05.812703 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:05.812766 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:05.817267 (+ 4501us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:05.817320 (+ 53us) server_negotiation.cc:299] Negotiation successful
0806 22:51:05.817369 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":617,"thread_start_us":76,"threads_started":1}
I20260806 22:51:05.817899 2258 tablet_bootstrap.cc:654] T 8c53cdccd609434d84bd5c1ba5d772ba P 6b923d20c1c2436d9fa8761461ef9e50: Neither blocks nor log segments found. Creating new log.
I20260806 22:51:05.818079 2257 tablet_bootstrap.cc:654] T 8c53cdccd609434d84bd5c1ba5d772ba P c5d439a6aafd41cdb6e2dc34afe07409: Neither blocks nor log segments found. Creating new log.
I20260806 22:51:05.818126 2147 tablet_service.cc:1467] Processing CreateTablet for tablet 8c53cdccd609434d84bd5c1ba5d772ba (DEFAULT_TABLE table=test-table [id=2d0c29ccc40e40f8a291ec138c71f63f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:51:05.818285 2258 log.cc:824] T 8c53cdccd609434d84bd5c1ba5d772ba P 6b923d20c1c2436d9fa8761461ef9e50: Log is configured to *not* fsync() on all Append() calls
I20260806 22:51:05.818446 2257 log.cc:824] T 8c53cdccd609434d84bd5c1ba5d772ba P c5d439a6aafd41cdb6e2dc34afe07409: Log is configured to *not* fsync() on all Append() calls
I20260806 22:51:05.818499 2147 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c53cdccd609434d84bd5c1ba5d772ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:51:05.819213 2258 tablet_bootstrap.cc:492] T 8c53cdccd609434d84bd5c1ba5d772ba P 6b923d20c1c2436d9fa8761461ef9e50: No bootstrap required, opened a new log
I20260806 22:51:05.819247 2257 tablet_bootstrap.cc:492] T 8c53cdccd609434d84bd5c1ba5d772ba P c5d439a6aafd41cdb6e2dc34afe07409: No bootstrap required, opened a new log
I20260806 22:51:05.819295 2258 ts_tablet_manager.cc:1397] T 8c53cdccd609434d84bd5c1ba5d772ba P 6b923d20c1c2436d9fa8761461ef9e50: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:51:05.819422 2257 ts_tablet_manager.cc:1397] T 8c53cdccd609434d84bd5c1ba5d772ba P c5d439a6aafd41cdb6e2dc34afe07409: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:51:05.821084 2262 tablet_bootstrap.cc:492] T 8c53cdccd609434d84bd5c1ba5d772ba P e8a91b570d0d4a26bbfaebfacd50335b: Bootstrap starting.
I20260806 22:51:05.821653 2262 tablet_bootstrap.cc:654] T 8c53cdccd609434d84bd5c1ba5d772ba P e8a91b570d0d4a26bbfaebfacd50335b: Neither blocks nor log segments found. Creating new log.
I20260806 22:51:05.821949 2262 log.cc:824] T 8c53cdccd609434d84bd5c1ba5d772ba P e8a91b570d0d4a26bbfaebfacd50335b: Log is configured to *not* fsync() on all Append() calls
I20260806 22:51:05.822078 2258 raft_consensus.cc:348] T 8c53cdccd609434d84bd5c1ba5d772ba P 6b923d20c1c2436d9fa8761461ef9e50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a91b570d0d4a26bbfaebfacd50335b" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 43311 } } peers { permanent_uuid: "c5d439a6aafd41cdb6e2dc34afe07409" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 40159 } } peers { permanent_uuid: "6b923d20c1c2436d9fa8761461ef9e50" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 46547 } }
I20260806 22:51:05.822259 2258 raft_consensus.cc:374] T 8c53cdccd609434d84bd5c1ba5d772ba P 6b923d20c1c2436d9fa8761461ef9e50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:51:05.822193 2257 raft_consensus.cc:348] T 8c53cdccd609434d84bd5c1ba5d772ba P c5d439a6aafd41cdb6e2dc34afe07409 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a91b570d0d4a26bbfaebfacd50335b" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 43311 } } peers { permanent_uuid: "c5d439a6aafd41cdb6e2dc34afe07409" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 40159 } } peers { permanent_uuid: "6b923d20c1c2436d9fa8761461ef9e50" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 46547 } }
I20260806 22:51:05.822306 2258 raft_consensus.cc:729] T 8c53cdccd609434d84bd5c1ba5d772ba P 6b923d20c1c2436d9fa8761461ef9e50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b923d20c1c2436d9fa8761461ef9e50, State: Initialized, Role: FOLLOWER
I20260806 22:51:05.822331 2257 raft_consensus.cc:374] T 8c53cdccd609434d84bd5c1ba5d772ba P c5d439a6aafd41cdb6e2dc34afe07409 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:51:05.822364 2257 raft_consensus.cc:729] T 8c53cdccd609434d84bd5c1ba5d772ba P c5d439a6aafd41cdb6e2dc34afe07409 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5d439a6aafd41cdb6e2dc34afe07409, State: Initialized, Role: FOLLOWER
I20260806 22:51:05.822510 2257 consensus_queue.cc:260] T 8c53cdccd609434d84bd5c1ba5d772ba P c5d439a6aafd41cdb6e2dc34afe07409 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a91b570d0d4a26bbfaebfacd50335b" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 43311 } } peers { permanent_uuid: "c5d439a6aafd41cdb6e2dc34afe07409" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 40159 } } peers { permanent_uuid: "6b923d20c1c2436d9fa8761461ef9e50" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 46547 } }
I20260806 22:51:05.822718 2262 tablet_bootstrap.cc:492] T 8c53cdccd609434d84bd5c1ba5d772ba P e8a91b570d0d4a26bbfaebfacd50335b: No bootstrap required, opened a new log
I20260806 22:51:05.822794 2262 ts_tablet_manager.cc:1397] T 8c53cdccd609434d84bd5c1ba5d772ba P e8a91b570d0d4a26bbfaebfacd50335b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:51:05.822530 2258 consensus_queue.cc:260] T 8c53cdccd609434d84bd5c1ba5d772ba P 6b923d20c1c2436d9fa8761461ef9e50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a91b570d0d4a26bbfaebfacd50335b" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 43311 } } peers { permanent_uuid: "c5d439a6aafd41cdb6e2dc34afe07409" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 40159 } } peers { permanent_uuid: "6b923d20c1c2436d9fa8761461ef9e50" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 46547 } }
I20260806 22:51:05.822866 2257 ts_tablet_manager.cc:1428] T 8c53cdccd609434d84bd5c1ba5d772ba P c5d439a6aafd41cdb6e2dc34afe07409: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:51:05.823010 1868 heartbeater.cc:503] Master 127.6.40.126:34025 was elected leader, sending a full tablet report...
I20260806 22:51:05.823010 2258 ts_tablet_manager.cc:1428] T 8c53cdccd609434d84bd5c1ba5d772ba P 6b923d20c1c2436d9fa8761461ef9e50: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260806 22:51:05.823112 2057 heartbeater.cc:503] Master 127.6.40.126:34025 was elected leader, sending a full tablet report...
I20260806 22:51:05.825169 2262 raft_consensus.cc:348] T 8c53cdccd609434d84bd5c1ba5d772ba P e8a91b570d0d4a26bbfaebfacd50335b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a91b570d0d4a26bbfaebfacd50335b" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 43311 } } peers { permanent_uuid: "c5d439a6aafd41cdb6e2dc34afe07409" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 40159 } } peers { permanent_uuid: "6b923d20c1c2436d9fa8761461ef9e50" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 46547 } }
I20260806 22:51:05.825309 2262 raft_consensus.cc:374] T 8c53cdccd609434d84bd5c1ba5d772ba P e8a91b570d0d4a26bbfaebfacd50335b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:51:05.825356 2262 raft_consensus.cc:729] T 8c53cdccd609434d84bd5c1ba5d772ba P e8a91b570d0d4a26bbfaebfacd50335b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8a91b570d0d4a26bbfaebfacd50335b, State: Initialized, Role: FOLLOWER
I20260806 22:51:05.825471 2262 consensus_queue.cc:260] T 8c53cdccd609434d84bd5c1ba5d772ba P e8a91b570d0d4a26bbfaebfacd50335b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a91b570d0d4a26bbfaebfacd50335b" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 43311 } } peers { permanent_uuid: "c5d439a6aafd41cdb6e2dc34afe07409" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 40159 } } peers { permanent_uuid: "6b923d20c1c2436d9fa8761461ef9e50" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 46547 } }
I20260806 22:51:05.825713 2262 ts_tablet_manager.cc:1428] T 8c53cdccd609434d84bd5c1ba5d772ba P e8a91b570d0d4a26bbfaebfacd50335b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:51:05.825775 2236 heartbeater.cc:503] Master 127.6.40.126:34025 was elected leader, sending a full tablet report...
W20260806 22:51:05.910854 2060 tablet.cc:2369] T 8c53cdccd609434d84bd5c1ba5d772ba P 6b923d20c1c2436d9fa8761461ef9e50: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:51:05.987396 2267 raft_consensus.cc:482] T 8c53cdccd609434d84bd5c1ba5d772ba P e8a91b570d0d4a26bbfaebfacd50335b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:51:05.987540 2267 raft_consensus.cc:504] T 8c53cdccd609434d84bd5c1ba5d772ba P e8a91b570d0d4a26bbfaebfacd50335b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a91b570d0d4a26bbfaebfacd50335b" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 43311 } } peers { permanent_uuid: "c5d439a6aafd41cdb6e2dc34afe07409" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 40159 } } peers { permanent_uuid: "6b923d20c1c2436d9fa8761461ef9e50" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 46547 } }
I20260806 22:51:05.987989 2267 leader_election.cc:288] T 8c53cdccd609434d84bd5c1ba5d772ba P e8a91b570d0d4a26bbfaebfacd50335b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c5d439a6aafd41cdb6e2dc34afe07409 (127.6.40.65:40159), 6b923d20c1c2436d9fa8761461ef9e50 (127.6.40.66:46547)
I20260806 22:51:05.992229 2256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:05.988323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.67:55591 (local address 127.6.40.66:46547)
0806 22:51:05.988377 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:05.988379 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:05.988389 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:51:05.988424 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:05.988425 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:05.988445 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:05.988498 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:51:05.988721 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:05.989505 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:05.991224 (+ 1719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:05.991790 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:05.991849 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:05.991857 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:05.992102 (+ 245us) server_negotiation.cc:299] Negotiation successful
0806 22:51:05.992153 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 22:51:05.992229 2251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:05.988278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.40.67:43343 (local address 127.6.40.65:40159)
0806 22:51:05.988329 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:05.988331 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:05.988552 (+ 221us) server_negotiation.cc:407] Connection header received
0806 22:51:05.988586 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:05.988591 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:05.988603 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:05.988646 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:51:05.988825 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:05.989538 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:05.991224 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:05.991766 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:05.991844 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:05.991855 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:05.992101 (+ 246us) server_negotiation.cc:299] Negotiation successful
0806 22:51:05.992153 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 22:51:05.992244 2239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:05.988211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.66:46547 (local address 127.6.40.67:55591)
0806 22:51:05.988286 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0806 22:51:05.988299 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:51:05.988346 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:51:05.988530 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:51:05.988531 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:51:05.988533 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:51:05.988616 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:05.988618 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:05.989643 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:05.989644 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:05.991117 (+ 1473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:05.991121 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:05.991880 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:05.991881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:05.991969 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:05.991976 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 22:51:05.992049 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 22:51:05.992114 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54,"mutex_wait_us":25}
I20260806 22:51:05.992251 2268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:05.988237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.65:40159 (local address 127.6.40.67:43343)
0806 22:51:05.988447 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0806 22:51:05.988455 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 22:51:05.988529 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:51:05.988677 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:51:05.988678 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:51:05.988680 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:51:05.988746 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:05.988747 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:05.989643 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:05.989644 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:05.991118 (+ 1474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:05.991121 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:05.991858 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:05.991860 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:05.991969 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:05.991976 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 22:51:05.992049 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 22:51:05.992111 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"mutex_wait_us":84,"thread_start_us":132,"threads_started":1}
I20260806 22:51:05.992746 1994 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c53cdccd609434d84bd5c1ba5d772ba" candidate_uuid: "e8a91b570d0d4a26bbfaebfacd50335b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b923d20c1c2436d9fa8761461ef9e50" is_pre_election: true
I20260806 22:51:05.992745 1806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c53cdccd609434d84bd5c1ba5d772ba" candidate_uuid: "e8a91b570d0d4a26bbfaebfacd50335b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5d439a6aafd41cdb6e2dc34afe07409" is_pre_election: true
I20260806 22:51:05.992969 1806 raft_consensus.cc:2454] T 8c53cdccd609434d84bd5c1ba5d772ba P c5d439a6aafd41cdb6e2dc34afe07409 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8a91b570d0d4a26bbfaebfacd50335b in term 0.
I20260806 22:51:05.992969 1994 raft_consensus.cc:2454] T 8c53cdccd609434d84bd5c1ba5d772ba P 6b923d20c1c2436d9fa8761461ef9e50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8a91b570d0d4a26bbfaebfacd50335b in term 0.
I20260806 22:51:05.993242 2086 leader_election.cc:302] T 8c53cdccd609434d84bd5c1ba5d772ba P e8a91b570d0d4a26bbfaebfacd50335b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b923d20c1c2436d9fa8761461ef9e50, e8a91b570d0d4a26bbfaebfacd50335b; no voters:
I20260806 22:51:05.993444 2267 raft_consensus.cc:2781] T 8c53cdccd609434d84bd5c1ba5d772ba P e8a91b570d0d4a26bbfaebfacd50335b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:51:05.993489 2267 raft_consensus.cc:482] T 8c53cdccd609434d84bd5c1ba5d772ba P e8a91b570d0d4a26bbfaebfacd50335b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:51:05.993505 2267 raft_consensus.cc:3037] T 8c53cdccd609434d84bd5c1ba5d772ba P e8a91b570d0d4a26bbfaebfacd50335b [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:51:05.994241 2267 raft_consensus.cc:504] T 8c53cdccd609434d84bd5c1ba5d772ba P e8a91b570d0d4a26bbfaebfacd50335b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a91b570d0d4a26bbfaebfacd50335b" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 43311 } } peers { permanent_uuid: "c5d439a6aafd41cdb6e2dc34afe07409" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 40159 } } peers { permanent_uuid: "6b923d20c1c2436d9fa8761461ef9e50" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 46547 } }
I20260806 22:51:05.994403 2267 leader_election.cc:288] T 8c53cdccd609434d84bd5c1ba5d772ba P e8a91b570d0d4a26bbfaebfacd50335b [CANDIDATE]: Term 1 election: Requested vote from peers c5d439a6aafd41cdb6e2dc34afe07409 (127.6.40.65:40159), 6b923d20c1c2436d9fa8761461ef9e50 (127.6.40.66:46547)
I20260806 22:51:05.994577 1806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c53cdccd609434d84bd5c1ba5d772ba" candidate_uuid: "e8a91b570d0d4a26bbfaebfacd50335b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5d439a6aafd41cdb6e2dc34afe07409"
I20260806 22:51:05.994601 1994 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c53cdccd609434d84bd5c1ba5d772ba" candidate_uuid: "e8a91b570d0d4a26bbfaebfacd50335b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b923d20c1c2436d9fa8761461ef9e50"
I20260806 22:51:05.994668 1806 raft_consensus.cc:3037] T 8c53cdccd609434d84bd5c1ba5d772ba P c5d439a6aafd41cdb6e2dc34afe07409 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:51:05.994675 1994 raft_consensus.cc:3037] T 8c53cdccd609434d84bd5c1ba5d772ba P 6b923d20c1c2436d9fa8761461ef9e50 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:51:05.995714 1994 raft_consensus.cc:2454] T 8c53cdccd609434d84bd5c1ba5d772ba P 6b923d20c1c2436d9fa8761461ef9e50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8a91b570d0d4a26bbfaebfacd50335b in term 1.
I20260806 22:51:05.995714 1806 raft_consensus.cc:2454] T 8c53cdccd609434d84bd5c1ba5d772ba P c5d439a6aafd41cdb6e2dc34afe07409 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8a91b570d0d4a26bbfaebfacd50335b in term 1.
I20260806 22:51:05.995925 2086 leader_election.cc:302] T 8c53cdccd609434d84bd5c1ba5d772ba P e8a91b570d0d4a26bbfaebfacd50335b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b923d20c1c2436d9fa8761461ef9e50, e8a91b570d0d4a26bbfaebfacd50335b; no voters:
I20260806 22:51:05.996080 2267 raft_consensus.cc:2781] T 8c53cdccd609434d84bd5c1ba5d772ba P e8a91b570d0d4a26bbfaebfacd50335b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:51:05.996229 2267 raft_consensus.cc:686] T 8c53cdccd609434d84bd5c1ba5d772ba P e8a91b570d0d4a26bbfaebfacd50335b [term 1 LEADER]: Becoming Leader. State: Replica: e8a91b570d0d4a26bbfaebfacd50335b, State: Running, Role: LEADER
I20260806 22:51:05.996357 2267 consensus_queue.cc:237] T 8c53cdccd609434d84bd5c1ba5d772ba P e8a91b570d0d4a26bbfaebfacd50335b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a91b570d0d4a26bbfaebfacd50335b" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 43311 } } peers { permanent_uuid: "c5d439a6aafd41cdb6e2dc34afe07409" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 40159 } } peers { permanent_uuid: "6b923d20c1c2436d9fa8761461ef9e50" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 46547 } }
I20260806 22:51:05.997283 1612 catalog_manager.cc:5495] T 8c53cdccd609434d84bd5c1ba5d772ba P e8a91b570d0d4a26bbfaebfacd50335b reported cstate change: term changed from 0 to 1, leader changed from <none> to e8a91b570d0d4a26bbfaebfacd50335b (127.6.40.67). New cstate: current_term: 1 leader_uuid: "e8a91b570d0d4a26bbfaebfacd50335b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a91b570d0d4a26bbfaebfacd50335b" member_type: VOTER last_known_addr { host: "127.6.40.67" port: 43311 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c5d439a6aafd41cdb6e2dc34afe07409" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 40159 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b923d20c1c2436d9fa8761461ef9e50" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 46547 } health_report { overall_health: UNKNOWN } } }
W20260806 22:51:06.006531 1869 tablet.cc:2369] T 8c53cdccd609434d84bd5c1ba5d772ba P c5d439a6aafd41cdb6e2dc34afe07409: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:51:06.038867 2238 tablet.cc:2369] T 8c53cdccd609434d84bd5c1ba5d772ba P e8a91b570d0d4a26bbfaebfacd50335b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:51:06.041245 2254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:06.039057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60160 (local address 127.6.40.67:43311)
0806 22:51:06.039102 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:06.039104 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:06.039114 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:51:06.039163 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:06.039163 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:06.039181 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:06.039248 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:51:06.039409 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:06.039871 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:06.040565 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:06.040746 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:06.040798 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:06.040959 (+ 161us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:51:06.041056 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:51:06.041121 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:06.041155 (+ 34us) server_negotiation.cc:299] Negotiation successful
0806 22:51:06.041170 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 22:51:06.043094 1806 raft_consensus.cc:1261] T 8c53cdccd609434d84bd5c1ba5d772ba P c5d439a6aafd41cdb6e2dc34afe07409 [term 1 FOLLOWER]: Refusing update from remote peer e8a91b570d0d4a26bbfaebfacd50335b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:51:06.043133 1994 raft_consensus.cc:1261] T 8c53cdccd609434d84bd5c1ba5d772ba P 6b923d20c1c2436d9fa8761461ef9e50 [term 1 FOLLOWER]: Refusing update from remote peer e8a91b570d0d4a26bbfaebfacd50335b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:51:06.043527 2269 consensus_queue.cc:1035] T 8c53cdccd609434d84bd5c1ba5d772ba P e8a91b570d0d4a26bbfaebfacd50335b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5d439a6aafd41cdb6e2dc34afe07409" member_type: VOTER last_known_addr { host: "127.6.40.65" port: 40159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:51:06.043666 2267 consensus_queue.cc:1035] T 8c53cdccd609434d84bd5c1ba5d772ba P e8a91b570d0d4a26bbfaebfacd50335b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b923d20c1c2436d9fa8761461ef9e50" member_type: VOTER last_known_addr { host: "127.6.40.66" port: 46547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:51:06.046317 2274 mvcc.cc:205] Tried to move back new op lower bound from 7315688104109563904 to 7315688103921807360. Current Snapshot: MvccSnapshot[applied={T|T < 7315688104109563904}]
I20260806 22:51:06.047034 2273 mvcc.cc:205] Tried to move back new op lower bound from 7315688104109563904 to 7315688103921807360. Current Snapshot: MvccSnapshot[applied={T|T < 7315688104109563904 or (T in {7315688104109563904})}]
I20260806 22:51:06.048462 1613 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34094:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:51:06.048564 1613 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34094:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:51:06.049332 1613 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 721ea693df0846a5bfbece38ac1aa18f: Sending DeleteTablet for 3 replicas of tablet 8c53cdccd609434d84bd5c1ba5d772ba
I20260806 22:51:06.049690 2147 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c53cdccd609434d84bd5c1ba5d772ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:51:06 UTC) from {username='slave'} at 127.0.0.1:60152
I20260806 22:51:06.049744 1963 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c53cdccd609434d84bd5c1ba5d772ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:51:06 UTC) from {username='slave'} at 127.0.0.1:40820
I20260806 22:51:06.049866 1780 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c53cdccd609434d84bd5c1ba5d772ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:51:06 UTC) from {username='slave'} at 127.0.0.1:49554
I20260806 22:51:06.050009 2283 tablet_replica.cc:330] T 8c53cdccd609434d84bd5c1ba5d772ba P e8a91b570d0d4a26bbfaebfacd50335b: stopping tablet replica
I20260806 22:51:06.050117 2283 raft_consensus.cc:2229] T 8c53cdccd609434d84bd5c1ba5d772ba P e8a91b570d0d4a26bbfaebfacd50335b [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:51:06.050130 2285 tablet_replica.cc:330] T 8c53cdccd609434d84bd5c1ba5d772ba P c5d439a6aafd41cdb6e2dc34afe07409: stopping tablet replica
I20260806 22:51:06.050248 2283 raft_consensus.cc:2258] T 8c53cdccd609434d84bd5c1ba5d772ba P e8a91b570d0d4a26bbfaebfacd50335b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:51:06.050384 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 1698
I20260806 22:51:06.050420 2284 tablet_replica.cc:330] T 8c53cdccd609434d84bd5c1ba5d772ba P 6b923d20c1c2436d9fa8761461ef9e50: stopping tablet replica
I20260806 22:51:06.050525 2284 raft_consensus.cc:2229] T 8c53cdccd609434d84bd5c1ba5d772ba P 6b923d20c1c2436d9fa8761461ef9e50 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:51:06.050621 2284 raft_consensus.cc:2258] T 8c53cdccd609434d84bd5c1ba5d772ba P 6b923d20c1c2436d9fa8761461ef9e50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:51:06.050999 2283 ts_tablet_manager.cc:1905] T 8c53cdccd609434d84bd5c1ba5d772ba P e8a91b570d0d4a26bbfaebfacd50335b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:51:06.052752 2283 ts_tablet_manager.cc:1918] T 8c53cdccd609434d84bd5c1ba5d772ba P e8a91b570d0d4a26bbfaebfacd50335b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:51:06.052852 2283 log.cc:1196] T 8c53cdccd609434d84bd5c1ba5d772ba P e8a91b570d0d4a26bbfaebfacd50335b: Deleting WAL directory at /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056405724609-6305-0/minicluster-data/ts-2/wal/wals/8c53cdccd609434d84bd5c1ba5d772ba
I20260806 22:51:06.053124 2283 ts_tablet_manager.cc:1939] T 8c53cdccd609434d84bd5c1ba5d772ba P e8a91b570d0d4a26bbfaebfacd50335b: Deleting consensus metadata
I20260806 22:51:06.053539 1595 catalog_manager.cc:4841] TS e8a91b570d0d4a26bbfaebfacd50335b (127.6.40.67:43311): tablet 8c53cdccd609434d84bd5c1ba5d772ba (table test-table [id=2d0c29ccc40e40f8a291ec138c71f63f]) successfully deleted
I20260806 22:51:06.053944 2284 ts_tablet_manager.cc:1905] T 8c53cdccd609434d84bd5c1ba5d772ba P 6b923d20c1c2436d9fa8761461ef9e50: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:51:06.055354 2284 ts_tablet_manager.cc:1918] T 8c53cdccd609434d84bd5c1ba5d772ba P 6b923d20c1c2436d9fa8761461ef9e50: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:51:06.055441 2284 log.cc:1196] T 8c53cdccd609434d84bd5c1ba5d772ba P 6b923d20c1c2436d9fa8761461ef9e50: Deleting WAL directory at /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056405724609-6305-0/minicluster-data/ts-1/wal/wals/8c53cdccd609434d84bd5c1ba5d772ba
I20260806 22:51:06.055696 2284 ts_tablet_manager.cc:1939] T 8c53cdccd609434d84bd5c1ba5d772ba P 6b923d20c1c2436d9fa8761461ef9e50: Deleting consensus metadata
I20260806 22:51:06.056105 1595 catalog_manager.cc:4841] TS 6b923d20c1c2436d9fa8761461ef9e50 (127.6.40.66:46547): tablet 8c53cdccd609434d84bd5c1ba5d772ba (table test-table [id=2d0c29ccc40e40f8a291ec138c71f63f]) successfully deleted
W20260806 22:51:06.063274 1596 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.40.65:40159 (error 108)
I20260806 22:51:06.063498 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 1874
I20260806 22:51:06.064306 2252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:06.063693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.65:40159 (local address 127.0.0.1:49566)
0806 22:51:06.064212 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0806 22:51:06.064268 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.65:40159: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":494}
W20260806 22:51:06.067173 1596 catalog_manager.cc:4584] TS c5d439a6aafd41cdb6e2dc34afe07409 (127.6.40.65:40159): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8c53cdccd609434d84bd5c1ba5d772ba: Network error: Client connection negotiation failed: client connection to 127.6.40.65:40159: connect: Connection refused (error 111)
I20260806 22:51:06.074436 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 2063
I20260806 22:51:06.083765 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 1576
2026-08-06T22:51:06Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (948 ms)
[----------] 6 tests from EncryptionPolicyTest (9110 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056405724609-6305-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:51:06 dist-test-slave-zh2d krb5kdc[2289](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:51:06 dist-test-slave-zh2d krb5kdc[2289](info): set up 2 sockets
Aug 06 22:51:06 dist-test-slave-zh2d krb5kdc[2289](info): commencing operation
krb5kdc: starting...
W20260806 22:51:07.298215 6305 mini_kdc.cc:121] Time spent starting KDC: real 1.189s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056405724609-6305-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056405724609-6305-0/krb5kdc/test-user.keytab.
Aug 06 22:51:07 dist-test-slave-zh2d krb5kdc[2289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056667, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:51:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:51:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056405724609-6305-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056405724609-6305-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 22:51:07.424108 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056405724609-6305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056405724609-6305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056405724609-6305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42305
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056405724609-6305-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42305
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:51:07.508487 2308 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:51:07.508746 2308 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:51:07.508809 2308 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:51:07.510052 2308 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:51:07.510098 2308 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:51:07.510113 2308 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:51:07.510125 2308 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:51:07.510138 2308 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:51:07.511695 2308 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056405724609-6305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056405724609-6305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42305
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056405724609-6305-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42305
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2308
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056405724609-6305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:51:07.511977 2308 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:51:07.512229 2308 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:51:07.517578 2316 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:51:07.517588 2318 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:51:07.517655 2308 server_base.cc:1034] running on GCE node
W20260806 22:51:07.517588 2315 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:51:07.518072 2308 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:51:07.518374 2308 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:51:07.519528 2308 hybrid_clock.cc:648] HybridClock initialized: now 1786056667519500 us; error 48 us; skew 500 ppm
Aug 06 22:51:07 dist-test-slave-zh2d krb5kdc[2289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056667, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:51:07.528494 2308 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 22:51:07.529109 2308 webserver.cc:460] Webserver started at http://127.0.0.1:42805/ using document root <none> and password file <none>
I20260806 22:51:07.529376 2308 fs_manager.cc:359] Metadata directory not provided
I20260806 22:51:07.529429 2308 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:51:07.529541 2308 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:51:07.530727 2308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056405724609-6305-0/minicluster-data/master-0/data/instance:
uuid: "5bda09a2903f474ba8ce4dee97cee2e7"
format_stamp: "Formatted at 2026-08-06 22:51:07 on dist-test-slave-zh2d"
server_key: "2f0baf33b67dc86a0d73bcf368225fad"
server_key_iv: "bc3da37cc7d1c2e32b15fe814bd38da4"
server_key_version: "encryptionkey@0"
I20260806 22:51:07.531090 2308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056405724609-6305-0/minicluster-data/master-0/wal/instance:
uuid: "5bda09a2903f474ba8ce4dee97cee2e7"
format_stamp: "Formatted at 2026-08-06 22:51:07 on dist-test-slave-zh2d"
server_key: "2f0baf33b67dc86a0d73bcf368225fad"
server_key_iv: "bc3da37cc7d1c2e32b15fe814bd38da4"
server_key_version: "encryptionkey@0"
I20260806 22:51:07.532653 2308 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:51:07.533712 2328 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:51:07.533989 2308 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:51:07.534126 2308 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056405724609-6305-0/minicluster-data/master-0/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056405724609-6305-0/minicluster-data/master-0/wal
uuid: "5bda09a2903f474ba8ce4dee97cee2e7"
format_stamp: "Formatted at 2026-08-06 22:51:07 on dist-test-slave-zh2d"
server_key: "2f0baf33b67dc86a0d73bcf368225fad"
server_key_iv: "bc3da37cc7d1c2e32b15fe814bd38da4"
server_key_version: "encryptionkey@0"
I20260806 22:51:07.534217 2308 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056405724609-6305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056405724609-6305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056405724609-6305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:51:07.563654 2308 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:51:07.566572 2308 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:51:07.566776 2308 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:51:07.571906 2308 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42305
I20260806 22:51:07.571950 2394 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42305 every 8 connection(s)
I20260806 22:51:07.572398 2308 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
I20260806 22:51:07.573575 2395 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:51:07.575968 2395 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5bda09a2903f474ba8ce4dee97cee2e7: Bootstrap starting.
I20260806 22:51:07.576571 2395 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5bda09a2903f474ba8ce4dee97cee2e7: Neither blocks nor log segments found. Creating new log.
I20260806 22:51:07.576916 2395 log.cc:824] T 00000000000000000000000000000000 P 5bda09a2903f474ba8ce4dee97cee2e7: Log is configured to *not* fsync() on all Append() calls
I20260806 22:51:07.577705 2395 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5bda09a2903f474ba8ce4dee97cee2e7: No bootstrap required, opened a new log
I20260806 22:51:07.579350 2395 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5bda09a2903f474ba8ce4dee97cee2e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bda09a2903f474ba8ce4dee97cee2e7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42305 } }
I20260806 22:51:07.579491 2395 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5bda09a2903f474ba8ce4dee97cee2e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:51:07.579514 2395 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5bda09a2903f474ba8ce4dee97cee2e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5bda09a2903f474ba8ce4dee97cee2e7, State: Initialized, Role: FOLLOWER
I20260806 22:51:07.579672 2395 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5bda09a2903f474ba8ce4dee97cee2e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bda09a2903f474ba8ce4dee97cee2e7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42305 } }
I20260806 22:51:07.579751 2395 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5bda09a2903f474ba8ce4dee97cee2e7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:51:07.579773 2395 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5bda09a2903f474ba8ce4dee97cee2e7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:51:07.579846 2395 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5bda09a2903f474ba8ce4dee97cee2e7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:51:07.579869 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 2308
I20260806 22:51:07.579993 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056405724609-6305-0/minicluster-data/master-0/wal/instance
I20260806 22:51:07.580204 6305 external_mini_cluster.cc:1428] Setting key 052185199c57e240275996d942087587
I20260806 22:51:07.580528 2395 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5bda09a2903f474ba8ce4dee97cee2e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bda09a2903f474ba8ce4dee97cee2e7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42305 } }
I20260806 22:51:07.580652 2395 leader_election.cc:302] T 00000000000000000000000000000000 P 5bda09a2903f474ba8ce4dee97cee2e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5bda09a2903f474ba8ce4dee97cee2e7; no voters:
I20260806 22:51:07.580897 2395 leader_election.cc:288] T 00000000000000000000000000000000 P 5bda09a2903f474ba8ce4dee97cee2e7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:51:07.580931 2402 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5bda09a2903f474ba8ce4dee97cee2e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:51:07.581092 2402 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5bda09a2903f474ba8ce4dee97cee2e7 [term 1 LEADER]: Becoming Leader. State: Replica: 5bda09a2903f474ba8ce4dee97cee2e7, State: Running, Role: LEADER
I20260806 22:51:07.581175 2402 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5bda09a2903f474ba8ce4dee97cee2e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bda09a2903f474ba8ce4dee97cee2e7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42305 } }
I20260806 22:51:07.581614 2402 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5bda09a2903f474ba8ce4dee97cee2e7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5bda09a2903f474ba8ce4dee97cee2e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bda09a2903f474ba8ce4dee97cee2e7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42305 } } }
I20260806 22:51:07.581692 2402 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5bda09a2903f474ba8ce4dee97cee2e7 [sys.catalog]: This master's current role is: LEADER
I20260806 22:51:07.581732 2395 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5bda09a2903f474ba8ce4dee97cee2e7 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:51:07.581707 2404 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5bda09a2903f474ba8ce4dee97cee2e7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5bda09a2903f474ba8ce4dee97cee2e7. Latest consensus state: current_term: 1 leader_uuid: "5bda09a2903f474ba8ce4dee97cee2e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bda09a2903f474ba8ce4dee97cee2e7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42305 } } }
I20260806 22:51:07.581835 2404 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5bda09a2903f474ba8ce4dee97cee2e7 [sys.catalog]: This master's current role is: LEADER
I20260806 22:51:07.582041 2406 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:51:07.582612 2406 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:51:07.584301 2406 catalog_manager.cc:1347] Generated new cluster ID: 0a52f93e1d244021b8be60ed81767c7b
I20260806 22:51:07.584357 2406 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 22:51:07 dist-test-slave-zh2d krb5kdc[2289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056667, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 22:51:07.594205 2403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:07.580758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59391 (local address 127.0.0.1:42305)
0806 22:51:07.581043 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:07.581054 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:07.581094 (+ 40us) server_negotiation.cc:407] Connection header received
0806 22:51:07.581212 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:07.581219 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:07.581279 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:07.581455 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:07.582577 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:07.584240 (+ 1663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:07.585513 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:07.586115 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:07.586218 (+ 103us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:07.589553 (+ 3335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:51:07.589557 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:51:07.589557 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:51:07.589570 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:51:07.591961 (+ 2391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:07.592468 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:07.592471 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:07.592472 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:07.592637 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:07.592934 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:07.592936 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:07.592936 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:07.593350 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:51:07.593673 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:07.593845 (+ 172us) server_negotiation.cc:299] Negotiation successful
0806 22:51:07.593967 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":229,"thread_start_us":138,"threads_started":1}
I20260806 22:51:07.595304 6305 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:51:07.603611 2406 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:51:07.604337 2406 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:51:07.604578 2403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:07.596789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52049 (local address 127.0.0.1:42305)
0806 22:51:07.596825 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:07.596827 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:07.596993 (+ 166us) server_negotiation.cc:407] Connection header received
0806 22:51:07.597025 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:07.597026 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:07.597039 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:07.597110 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:07.597681 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:07.598537 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:07.599592 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:07.599969 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:07.600048 (+ 79us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:07.601443 (+ 1395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:51:07.601445 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:51:07.601446 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:51:07.601453 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:51:07.603162 (+ 1709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:07.603697 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:07.603699 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:07.603699 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:07.603762 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:07.604018 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:07.604019 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:07.604019 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:07.604138 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:51:07.604258 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:07.604438 (+ 180us) server_negotiation.cc:299] Negotiation successful
0806 22:51:07.604505 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 22:51:07.610637 2406 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5bda09a2903f474ba8ce4dee97cee2e7: Generated new TSK 0
I20260806 22:51:07.610852 2406 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:51:07.617280 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 2308
W20260806 22:51:07.633425 2287 connection.cc:535] client connection to 127.0.0.1:42305 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42305): Connection reset by peer (error 104)
2026-08-06T22:51:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1538 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056405724609-6305-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:51:07 dist-test-slave-zh2d krb5kdc[2437](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:51:07 dist-test-slave-zh2d krb5kdc[2437](info): set up 2 sockets
Aug 06 22:51:07 dist-test-slave-zh2d krb5kdc[2437](info): commencing operation
krb5kdc: starting...
W20260806 22:51:08.850942 6305 mini_kdc.cc:121] Time spent starting KDC: real 1.203s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056405724609-6305-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056405724609-6305-0/krb5kdc/test-user.keytab.
Aug 06 22:51:08 dist-test-slave-zh2d krb5kdc[2437](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056668, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:51:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:51:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056405724609-6305-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056405724609-6305-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 22:51:08.981675 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056405724609-6305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056405724609-6305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056405724609-6305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35673
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056405724609-6305-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35673
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:51:09.065296 2456 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:51:09.065539 2456 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:51:09.065579 2456 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:51:09.066726 2456 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:51:09.066795 2456 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:51:09.066819 2456 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:51:09.066841 2456 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:51:09.066869 2456 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:51:09.068388 2456 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056405724609-6305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056405724609-6305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35673
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056405724609-6305-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35673
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2456
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056405724609-6305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:51:09.068679 2456 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:51:09.068936 2456 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:51:09.073875 2464 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:51:09.073882 2466 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:51:09.073890 2468 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:51:09.074126 2456 server_base.cc:1034] running on GCE node
I20260806 22:51:09.074432 2456 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:51:09.074728 2456 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:51:09.075891 2456 hybrid_clock.cc:648] HybridClock initialized: now 1786056669075873 us; error 46 us; skew 500 ppm
Aug 06 22:51:09 dist-test-slave-zh2d krb5kdc[2437](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056669, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:51:09.078574 2456 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 22:51:09.079025 2456 webserver.cc:460] Webserver started at http://127.0.0.1:37843/ using document root <none> and password file <none>
I20260806 22:51:09.079284 2456 fs_manager.cc:359] Metadata directory not provided
I20260806 22:51:09.079365 2456 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:51:09.079512 2456 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:51:09.080552 2456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056405724609-6305-0/minicluster-data/master-0/data/instance:
uuid: "e7416f45863343ab8abbc81c04f72e60"
format_stamp: "Formatted at 2026-08-06 22:51:09 on dist-test-slave-zh2d"
server_key: "3ff6d77bf258a487c8c7ba593b788d77"
server_key_iv: "bbbb0b1dd7091e45975ed9706577aafd"
server_key_version: "encryptionkey@0"
I20260806 22:51:09.080987 2456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056405724609-6305-0/minicluster-data/master-0/wal/instance:
uuid: "e7416f45863343ab8abbc81c04f72e60"
format_stamp: "Formatted at 2026-08-06 22:51:09 on dist-test-slave-zh2d"
server_key: "3ff6d77bf258a487c8c7ba593b788d77"
server_key_iv: "bbbb0b1dd7091e45975ed9706577aafd"
server_key_version: "encryptionkey@0"
I20260806 22:51:09.082397 2456 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:51:09.083175 2478 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:51:09.083400 2456 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:51:09.083511 2456 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056405724609-6305-0/minicluster-data/master-0/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056405724609-6305-0/minicluster-data/master-0/wal
uuid: "e7416f45863343ab8abbc81c04f72e60"
format_stamp: "Formatted at 2026-08-06 22:51:09 on dist-test-slave-zh2d"
server_key: "3ff6d77bf258a487c8c7ba593b788d77"
server_key_iv: "bbbb0b1dd7091e45975ed9706577aafd"
server_key_version: "encryptionkey@0"
I20260806 22:51:09.083593 2456 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056405724609-6305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056405724609-6305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056405724609-6305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:51:09.097743 2456 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:51:09.099746 2456 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:51:09.099917 2456 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:51:09.104399 2456 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35673
I20260806 22:51:09.104406 2555 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35673 every 8 connection(s)
I20260806 22:51:09.104789 2456 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
I20260806 22:51:09.105816 2556 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:51:09.107050 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 2456
I20260806 22:51:09.107187 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056405724609-6305-0/minicluster-data/master-0/wal/instance
I20260806 22:51:09.107396 6305 external_mini_cluster.cc:1428] Setting key 15dcfd51d8728eade2ed90731152a75d
I20260806 22:51:09.108608 2556 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7416f45863343ab8abbc81c04f72e60: Bootstrap starting.
I20260806 22:51:09.109314 2556 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e7416f45863343ab8abbc81c04f72e60: Neither blocks nor log segments found. Creating new log.
I20260806 22:51:09.109614 2556 log.cc:824] T 00000000000000000000000000000000 P e7416f45863343ab8abbc81c04f72e60: Log is configured to *not* fsync() on all Append() calls
I20260806 22:51:09.110412 2556 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7416f45863343ab8abbc81c04f72e60: No bootstrap required, opened a new log
I20260806 22:51:09.112709 2556 raft_consensus.cc:348] T 00000000000000000000000000000000 P e7416f45863343ab8abbc81c04f72e60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7416f45863343ab8abbc81c04f72e60" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35673 } }
I20260806 22:51:09.112914 2556 raft_consensus.cc:374] T 00000000000000000000000000000000 P e7416f45863343ab8abbc81c04f72e60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:51:09.113021 2556 raft_consensus.cc:729] T 00000000000000000000000000000000 P e7416f45863343ab8abbc81c04f72e60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7416f45863343ab8abbc81c04f72e60, State: Initialized, Role: FOLLOWER
I20260806 22:51:09.113243 2559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:09.107950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43145 (local address 127.0.0.1:35673)
0806 22:51:09.108122 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:09.108130 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:09.108155 (+ 25us) server_negotiation.cc:407] Connection header received
0806 22:51:09.108555 (+ 400us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:09.108561 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:09.108607 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:09.108776 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:09.109046 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:09.110177 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:09.111263 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:09.111679 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:09.111762 (+ 83us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:09.112534 (+ 772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:51:09.112538 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:51:09.112538 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:51:09.112706 (+ 168us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:51:09.112736 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:51:09.112746 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:51:09.112808 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:09.112986 (+ 178us) server_negotiation.cc:299] Negotiation successful
0806 22:51:09.113017 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":79,"threads_started":1}
I20260806 22:51:09.113220 2556 consensus_queue.cc:260] T 00000000000000000000000000000000 P e7416f45863343ab8abbc81c04f72e60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7416f45863343ab8abbc81c04f72e60" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35673 } }
I20260806 22:51:09.113363 2556 raft_consensus.cc:388] T 00000000000000000000000000000000 P e7416f45863343ab8abbc81c04f72e60 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:51:09.113415 2556 raft_consensus.cc:482] T 00000000000000000000000000000000 P e7416f45863343ab8abbc81c04f72e60 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:51:09.113492 2556 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e7416f45863343ab8abbc81c04f72e60 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:51:09.114212 6305 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:51:09.114195 2556 raft_consensus.cc:504] T 00000000000000000000000000000000 P e7416f45863343ab8abbc81c04f72e60 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7416f45863343ab8abbc81c04f72e60" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35673 } }
I20260806 22:51:09.114312 2556 leader_election.cc:302] T 00000000000000000000000000000000 P e7416f45863343ab8abbc81c04f72e60 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e7416f45863343ab8abbc81c04f72e60; no voters:
I20260806 22:51:09.114490 2556 leader_election.cc:288] T 00000000000000000000000000000000 P e7416f45863343ab8abbc81c04f72e60 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:51:09.114540 2562 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e7416f45863343ab8abbc81c04f72e60 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:51:09.114691 2562 raft_consensus.cc:686] T 00000000000000000000000000000000 P e7416f45863343ab8abbc81c04f72e60 [term 1 LEADER]: Becoming Leader. State: Replica: e7416f45863343ab8abbc81c04f72e60, State: Running, Role: LEADER
I20260806 22:51:09.114732 2556 sys_catalog.cc:564] T 00000000000000000000000000000000 P e7416f45863343ab8abbc81c04f72e60 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:51:09.114861 2562 consensus_queue.cc:237] T 00000000000000000000000000000000 P e7416f45863343ab8abbc81c04f72e60 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7416f45863343ab8abbc81c04f72e60" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35673 } }
I20260806 22:51:09.115249 2564 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7416f45863343ab8abbc81c04f72e60 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e7416f45863343ab8abbc81c04f72e60" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7416f45863343ab8abbc81c04f72e60" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35673 } } }
I20260806 22:51:09.115319 2564 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7416f45863343ab8abbc81c04f72e60 [sys.catalog]: This master's current role is: LEADER
I20260806 22:51:09.116109 2575 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:51:09.116318 2563 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7416f45863343ab8abbc81c04f72e60 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e7416f45863343ab8abbc81c04f72e60. Latest consensus state: current_term: 1 leader_uuid: "e7416f45863343ab8abbc81c04f72e60" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7416f45863343ab8abbc81c04f72e60" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35673 } } }
I20260806 22:51:09.116520 2563 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7416f45863343ab8abbc81c04f72e60 [sys.catalog]: This master's current role is: LEADER
I20260806 22:51:09.116616 2575 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:51:09.118229 2575 catalog_manager.cc:1347] Generated new cluster ID: ed0d4be0f37b4a749fbeb737a2d7e32f
I20260806 22:51:09.118280 2575 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:51:09.119925 2559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:09.115974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53837 (local address 127.0.0.1:35673)
0806 22:51:09.116014 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:09.116016 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:09.116668 (+ 652us) server_negotiation.cc:407] Connection header received
0806 22:51:09.116752 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:09.116753 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:09.116763 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:09.116826 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:09.117007 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:09.117560 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:09.118588 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:09.118877 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:09.118957 (+ 80us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:09.119677 (+ 720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:51:09.119678 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:51:09.119679 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:51:09.119683 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:51:09.119695 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:51:09.119699 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:51:09.119752 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:09.119850 (+ 98us) server_negotiation.cc:299] Negotiation successful
0806 22:51:09.119859 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 22:51:09.127477 2575 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:51:09.127995 2575 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:51:09.131047 2575 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e7416f45863343ab8abbc81c04f72e60: Generated new TSK 0
I20260806 22:51:09.131223 2575 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:51:09.136134 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 2456
W20260806 22:51:09.153614 2435 connection.cc:535] client connection to 127.0.0.1:35673 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35673): Connection reset by peer (error 104)
2026-08-06T22:51:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1519 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056405724609-6305-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:51:09 dist-test-slave-zh2d krb5kdc[2599](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:51:09 dist-test-slave-zh2d krb5kdc[2599](info): set up 2 sockets
Aug 06 22:51:09 dist-test-slave-zh2d krb5kdc[2599](info): commencing operation
krb5kdc: starting...
W20260806 22:51:10.416302 6305 mini_kdc.cc:121] Time spent starting KDC: real 1.249s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056405724609-6305-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056405724609-6305-0/krb5kdc/test-user.keytab.
Aug 06 22:51:10 dist-test-slave-zh2d krb5kdc[2599](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056670, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:51:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:51:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056405724609-6305-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056405724609-6305-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 22:51:10.544364 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056405724609-6305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056405724609-6305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056405724609-6305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36935
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056405724609-6305-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36935
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:51:10.626915 2613 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:51:10.627125 2613 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:51:10.627151 2613 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:51:10.628346 2613 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:51:10.628391 2613 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:51:10.628405 2613 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:51:10.628443 2613 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:51:10.628470 2613 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:51:10.630018 2613 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056405724609-6305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056405724609-6305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36935
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056405724609-6305-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36935
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2613
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056405724609-6305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:51:10.630283 2613 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:51:10.630518 2613 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 22:51:10.635739 2613 server_base.cc:1034] running on GCE node
W20260806 22:51:10.635756 2619 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:51:10.635721 2618 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:51:10.635849 2621 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:51:10.636279 2613 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:51:10.636619 2613 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:51:10.637779 2613 hybrid_clock.cc:648] HybridClock initialized: now 1786056670637775 us; error 36 us; skew 500 ppm
Aug 06 22:51:10 dist-test-slave-zh2d krb5kdc[2599](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056670, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:51:10.640336 2613 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 22:51:10.640822 2613 webserver.cc:460] Webserver started at http://127.0.0.1:41795/ using document root <none> and password file <none>
I20260806 22:51:10.641119 2613 fs_manager.cc:359] Metadata directory not provided
I20260806 22:51:10.641191 2613 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:51:10.641340 2613 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:51:10.642387 2613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056405724609-6305-0/minicluster-data/master-0/data/instance:
uuid: "350dbf930a8042e49181e1f9b56c3224"
format_stamp: "Formatted at 2026-08-06 22:51:10 on dist-test-slave-zh2d"
server_key: "f3193a991bdf2aa7aa1aa3f5d71c62aa"
server_key_iv: "3280076e876917d2f7a079eb7a09184b"
server_key_version: "encryptionkey@0"
I20260806 22:51:10.642794 2613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056405724609-6305-0/minicluster-data/master-0/wal/instance:
uuid: "350dbf930a8042e49181e1f9b56c3224"
format_stamp: "Formatted at 2026-08-06 22:51:10 on dist-test-slave-zh2d"
server_key: "f3193a991bdf2aa7aa1aa3f5d71c62aa"
server_key_iv: "3280076e876917d2f7a079eb7a09184b"
server_key_version: "encryptionkey@0"
I20260806 22:51:10.644218 2613 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:51:10.645038 2628 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:51:10.645211 2613 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:51:10.645320 2613 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056405724609-6305-0/minicluster-data/master-0/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056405724609-6305-0/minicluster-data/master-0/wal
uuid: "350dbf930a8042e49181e1f9b56c3224"
format_stamp: "Formatted at 2026-08-06 22:51:10 on dist-test-slave-zh2d"
server_key: "f3193a991bdf2aa7aa1aa3f5d71c62aa"
server_key_iv: "3280076e876917d2f7a079eb7a09184b"
server_key_version: "encryptionkey@0"
I20260806 22:51:10.645393 2613 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056405724609-6305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056405724609-6305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056405724609-6305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:51:10.648654 2613 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:51:10.650954 2613 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:51:10.651113 2613 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:51:10.655328 2613 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36935
I20260806 22:51:10.655359 2705 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36935 every 8 connection(s)
I20260806 22:51:10.655683 2613 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
I20260806 22:51:10.656821 2706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:51:10.659251 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 2613
I20260806 22:51:10.659392 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056405724609-6305-0/minicluster-data/master-0/wal/instance
I20260806 22:51:10.659649 6305 external_mini_cluster.cc:1428] Setting key d93310b331f5008d803089dffd364880
I20260806 22:51:10.659969 2706 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 350dbf930a8042e49181e1f9b56c3224: Bootstrap starting.
I20260806 22:51:10.660755 2706 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 350dbf930a8042e49181e1f9b56c3224: Neither blocks nor log segments found. Creating new log.
I20260806 22:51:10.661150 2706 log.cc:824] T 00000000000000000000000000000000 P 350dbf930a8042e49181e1f9b56c3224: Log is configured to *not* fsync() on all Append() calls
I20260806 22:51:10.662089 2706 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 350dbf930a8042e49181e1f9b56c3224: No bootstrap required, opened a new log
I20260806 22:51:10.662391 2712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:10.660208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59019 (local address 127.0.0.1:36935)
0806 22:51:10.660399 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:10.660409 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:10.660446 (+ 37us) server_negotiation.cc:407] Connection header received
0806 22:51:10.660982 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:10.660991 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:10.661022 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:10.661174 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:10.661878 (+ 704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:51:10.661883 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:51:10.661883 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:51:10.661889 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:51:10.661941 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:51:10.661958 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:51:10.662078 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:10.662117 (+ 39us) server_negotiation.cc:299] Negotiation successful
0806 22:51:10.662163 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":96,"threads_started":1}
I20260806 22:51:10.663516 6305 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:51:10.665336 2706 raft_consensus.cc:348] T 00000000000000000000000000000000 P 350dbf930a8042e49181e1f9b56c3224 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "350dbf930a8042e49181e1f9b56c3224" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36935 } }
I20260806 22:51:10.665596 2706 raft_consensus.cc:374] T 00000000000000000000000000000000 P 350dbf930a8042e49181e1f9b56c3224 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:51:10.665683 2706 raft_consensus.cc:729] T 00000000000000000000000000000000 P 350dbf930a8042e49181e1f9b56c3224 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 350dbf930a8042e49181e1f9b56c3224, State: Initialized, Role: FOLLOWER
I20260806 22:51:10.665858 2706 consensus_queue.cc:260] T 00000000000000000000000000000000 P 350dbf930a8042e49181e1f9b56c3224 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "350dbf930a8042e49181e1f9b56c3224" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36935 } }
I20260806 22:51:10.665983 2706 raft_consensus.cc:388] T 00000000000000000000000000000000 P 350dbf930a8042e49181e1f9b56c3224 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:51:10.666033 2706 raft_consensus.cc:482] T 00000000000000000000000000000000 P 350dbf930a8042e49181e1f9b56c3224 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:51:10.666101 2706 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 350dbf930a8042e49181e1f9b56c3224 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:51:10.666887 2712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:10.665513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34051 (local address 127.0.0.1:36935)
0806 22:51:10.665553 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:10.665555 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:10.665742 (+ 187us) server_negotiation.cc:407] Connection header received
0806 22:51:10.665799 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:10.665800 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:10.665806 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:10.665897 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:10.666656 (+ 759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:51:10.666657 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:51:10.666657 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:51:10.666661 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:51:10.666673 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:51:10.666676 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:51:10.666721 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:10.666793 (+ 72us) server_negotiation.cc:299] Negotiation successful
0806 22:51:10.666803 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 22:51:10.667055 2706 raft_consensus.cc:504] T 00000000000000000000000000000000 P 350dbf930a8042e49181e1f9b56c3224 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "350dbf930a8042e49181e1f9b56c3224" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36935 } }
I20260806 22:51:10.667234 2706 leader_election.cc:302] T 00000000000000000000000000000000 P 350dbf930a8042e49181e1f9b56c3224 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 350dbf930a8042e49181e1f9b56c3224; no voters:
I20260806 22:51:10.667449 2706 leader_election.cc:288] T 00000000000000000000000000000000 P 350dbf930a8042e49181e1f9b56c3224 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:51:10.667490 2719 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 350dbf930a8042e49181e1f9b56c3224 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:51:10.667665 2719 raft_consensus.cc:686] T 00000000000000000000000000000000 P 350dbf930a8042e49181e1f9b56c3224 [term 1 LEADER]: Becoming Leader. State: Replica: 350dbf930a8042e49181e1f9b56c3224, State: Running, Role: LEADER
I20260806 22:51:10.667800 2719 consensus_queue.cc:237] T 00000000000000000000000000000000 P 350dbf930a8042e49181e1f9b56c3224 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "350dbf930a8042e49181e1f9b56c3224" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36935 } }
I20260806 22:51:10.667802 2706 sys_catalog.cc:564] T 00000000000000000000000000000000 P 350dbf930a8042e49181e1f9b56c3224 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:51:10.668306 2720 sys_catalog.cc:455] T 00000000000000000000000000000000 P 350dbf930a8042e49181e1f9b56c3224 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "350dbf930a8042e49181e1f9b56c3224" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "350dbf930a8042e49181e1f9b56c3224" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36935 } } }
I20260806 22:51:10.668314 2721 sys_catalog.cc:455] T 00000000000000000000000000000000 P 350dbf930a8042e49181e1f9b56c3224 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 350dbf930a8042e49181e1f9b56c3224. Latest consensus state: current_term: 1 leader_uuid: "350dbf930a8042e49181e1f9b56c3224" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "350dbf930a8042e49181e1f9b56c3224" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36935 } } }
I20260806 22:51:10.668495 2720 sys_catalog.cc:458] T 00000000000000000000000000000000 P 350dbf930a8042e49181e1f9b56c3224 [sys.catalog]: This master's current role is: LEADER
I20260806 22:51:10.668506 2721 sys_catalog.cc:458] T 00000000000000000000000000000000 P 350dbf930a8042e49181e1f9b56c3224 [sys.catalog]: This master's current role is: LEADER
I20260806 22:51:10.669099 2729 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:51:10.669682 2729 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:51:10.671433 2729 catalog_manager.cc:1347] Generated new cluster ID: 2c299ffa57e648c2b5fdec11fc613db5
I20260806 22:51:10.671499 2729 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:51:10.693347 2729 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:51:10.693542 2729 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:51:10.699427 2729 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 350dbf930a8042e49181e1f9b56c3224: Generated new TSK 0
I20260806 22:51:10.699658 2729 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:51:10.703884 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 2613
2026-08-06T22:51:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1559 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056405724609-6305-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:51:10 dist-test-slave-zh2d krb5kdc[2746](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:51:10 dist-test-slave-zh2d krb5kdc[2746](info): set up 2 sockets
Aug 06 22:51:10 dist-test-slave-zh2d krb5kdc[2746](info): commencing operation
krb5kdc: starting...
W20260806 22:51:11.918907 6305 mini_kdc.cc:121] Time spent starting KDC: real 1.192s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056405724609-6305-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056405724609-6305-0/krb5kdc/test-user.keytab.
Aug 06 22:51:12 dist-test-slave-zh2d krb5kdc[2746](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056672, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:51:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:51:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056405724609-6305-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056405724609-6305-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 22:51:12.045681 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056405724609-6305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056405724609-6305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056405724609-6305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35119
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056405724609-6305-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:32881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35119
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:51:12.128916 2766 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:51:12.129120 2766 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:51:12.129144 2766 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:51:12.130224 2766 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:51:12.130264 2766 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:51:12.130278 2766 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:51:12.130290 2766 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:51:12.130303 2766 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:51:12.131721 2766 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:32881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056405724609-6305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056405724609-6305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35119
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056405724609-6305-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35119
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2766
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056405724609-6305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:51:12.131907 2766 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:51:12.132081 2766 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:51:12.137125 2777 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:51:12.137123 2775 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:51:12.137182 2774 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:51:12.137408 2766 server_base.cc:1034] running on GCE node
I20260806 22:51:12.137691 2766 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:51:12.137953 2766 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:51:12.139115 2766 hybrid_clock.cc:648] HybridClock initialized: now 1786056672139101 us; error 30 us; skew 500 ppm
Aug 06 22:51:12 dist-test-slave-zh2d krb5kdc[2746](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056672, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:51:12.142020 2766 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 22:51:12.142464 2766 webserver.cc:460] Webserver started at http://127.0.0.1:45795/ using document root <none> and password file <none>
I20260806 22:51:12.142693 2766 fs_manager.cc:359] Metadata directory not provided
I20260806 22:51:12.142760 2766 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:51:12.142907 2766 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:51:12.144052 2766 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056405724609-6305-0/minicluster-data/master-0/data/instance:
uuid: "4016383b19624ad3a2221b7f665caacf"
format_stamp: "Formatted at 2026-08-06 22:51:12 on dist-test-slave-zh2d"
server_key: "a36d791e228b2bbbfcb2d95741c4bcd9"
server_key_iv: "0217fde913bb506fa9bcc0662a3a7f08"
server_key_version: "encryptionkey@0"
I20260806 22:51:12.144426 2766 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056405724609-6305-0/minicluster-data/master-0/wal/instance:
uuid: "4016383b19624ad3a2221b7f665caacf"
format_stamp: "Formatted at 2026-08-06 22:51:12 on dist-test-slave-zh2d"
server_key: "a36d791e228b2bbbfcb2d95741c4bcd9"
server_key_iv: "0217fde913bb506fa9bcc0662a3a7f08"
server_key_version: "encryptionkey@0"
I20260806 22:51:12.145920 2766 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:51:12.146768 2786 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:51:12.146960 2766 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 22:51:12.147064 2766 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056405724609-6305-0/minicluster-data/master-0/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056405724609-6305-0/minicluster-data/master-0/wal
uuid: "4016383b19624ad3a2221b7f665caacf"
format_stamp: "Formatted at 2026-08-06 22:51:12 on dist-test-slave-zh2d"
server_key: "a36d791e228b2bbbfcb2d95741c4bcd9"
server_key_iv: "0217fde913bb506fa9bcc0662a3a7f08"
server_key_version: "encryptionkey@0"
I20260806 22:51:12.147147 2766 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056405724609-6305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056405724609-6305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056405724609-6305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:51:12.166038 2766 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:51:12.168421 2766 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:51:12.168592 2766 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:51:12.173494 2766 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35119
I20260806 22:51:12.173532 2859 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35119 every 8 connection(s)
I20260806 22:51:12.173907 2766 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
I20260806 22:51:12.174960 2860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:51:12.177353 2860 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4016383b19624ad3a2221b7f665caacf: Bootstrap starting.
I20260806 22:51:12.177910 2860 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4016383b19624ad3a2221b7f665caacf: Neither blocks nor log segments found. Creating new log.
I20260806 22:51:12.178164 2860 log.cc:824] T 00000000000000000000000000000000 P 4016383b19624ad3a2221b7f665caacf: Log is configured to *not* fsync() on all Append() calls
I20260806 22:51:12.178896 2860 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4016383b19624ad3a2221b7f665caacf: No bootstrap required, opened a new log
I20260806 22:51:12.180447 2860 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4016383b19624ad3a2221b7f665caacf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4016383b19624ad3a2221b7f665caacf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35119 } }
I20260806 22:51:12.180629 2860 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4016383b19624ad3a2221b7f665caacf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:51:12.180670 2860 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4016383b19624ad3a2221b7f665caacf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4016383b19624ad3a2221b7f665caacf, State: Initialized, Role: FOLLOWER
I20260806 22:51:12.180753 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 2766
I20260806 22:51:12.180886 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056405724609-6305-0/minicluster-data/master-0/wal/instance
I20260806 22:51:12.180860 2860 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4016383b19624ad3a2221b7f665caacf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4016383b19624ad3a2221b7f665caacf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35119 } }
I20260806 22:51:12.180944 2860 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4016383b19624ad3a2221b7f665caacf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:51:12.180989 2860 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4016383b19624ad3a2221b7f665caacf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:51:12.181046 2860 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4016383b19624ad3a2221b7f665caacf [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:51:12.181110 6305 external_mini_cluster.cc:1428] Setting key 8947533408a10191d698f37d6bee96f3
I20260806 22:51:12.181769 2860 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4016383b19624ad3a2221b7f665caacf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4016383b19624ad3a2221b7f665caacf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35119 } }
I20260806 22:51:12.181891 2860 leader_election.cc:302] T 00000000000000000000000000000000 P 4016383b19624ad3a2221b7f665caacf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4016383b19624ad3a2221b7f665caacf; no voters:
I20260806 22:51:12.182063 2860 leader_election.cc:288] T 00000000000000000000000000000000 P 4016383b19624ad3a2221b7f665caacf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:51:12.182094 2867 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4016383b19624ad3a2221b7f665caacf [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:51:12.182207 2867 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4016383b19624ad3a2221b7f665caacf [term 1 LEADER]: Becoming Leader. State: Replica: 4016383b19624ad3a2221b7f665caacf, State: Running, Role: LEADER
I20260806 22:51:12.182333 2860 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4016383b19624ad3a2221b7f665caacf [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:51:12.182368 2867 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4016383b19624ad3a2221b7f665caacf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4016383b19624ad3a2221b7f665caacf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35119 } }
I20260806 22:51:12.182796 2867 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4016383b19624ad3a2221b7f665caacf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4016383b19624ad3a2221b7f665caacf. Latest consensus state: current_term: 1 leader_uuid: "4016383b19624ad3a2221b7f665caacf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4016383b19624ad3a2221b7f665caacf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35119 } } }
I20260806 22:51:12.182890 2867 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4016383b19624ad3a2221b7f665caacf [sys.catalog]: This master's current role is: LEADER
I20260806 22:51:12.183073 2868 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4016383b19624ad3a2221b7f665caacf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4016383b19624ad3a2221b7f665caacf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4016383b19624ad3a2221b7f665caacf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35119 } } }
I20260806 22:51:12.183185 2871 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:51:12.183221 2868 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4016383b19624ad3a2221b7f665caacf [sys.catalog]: This master's current role is: LEADER
I20260806 22:51:12.183871 2871 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:51:12.185935 2871 catalog_manager.cc:1347] Generated new cluster ID: 3f1c9dfdd19143e4a12d45848b05ce1b
I20260806 22:51:12.186015 2871 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 22:51:12 dist-test-slave-zh2d krb5kdc[2746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056672, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 22:51:12.192116 2865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:12.181614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57471 (local address 127.0.0.1:35119)
0806 22:51:12.181823 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:12.181833 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:12.181867 (+ 34us) server_negotiation.cc:407] Connection header received
0806 22:51:12.181987 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:12.181994 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:12.182052 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:12.183052 (+ 1000us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:12.183156 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:12.184580 (+ 1424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:12.185496 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:12.186081 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:12.186186 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:12.188733 (+ 2547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:51:12.188737 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:51:12.188737 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:51:12.188749 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:51:12.190556 (+ 1807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:12.190956 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:12.190959 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:12.190959 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:12.191051 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:12.191266 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:12.191267 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:12.191268 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:12.191517 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:51:12.191685 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:12.191802 (+ 117us) server_negotiation.cc:299] Negotiation successful
0806 22:51:12.191954 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":130,"threads_started":1}
I20260806 22:51:12.193120 6305 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:51:12.200574 2865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:12.194509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45835 (local address 127.0.0.1:35119)
0806 22:51:12.194593 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:12.194594 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:12.194643 (+ 49us) server_negotiation.cc:407] Connection header received
0806 22:51:12.194714 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:12.194715 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:12.194725 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:12.194768 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:12.195298 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:12.195788 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:12.196518 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:12.196742 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:12.196818 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:12.198017 (+ 1199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:51:12.198018 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:51:12.198018 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:51:12.198025 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:51:12.199448 (+ 1423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:12.199801 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:12.199803 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:12.199803 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:12.199856 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:12.200078 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:12.200079 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:12.200080 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:12.200192 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:51:12.200295 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:12.200400 (+ 105us) server_negotiation.cc:299] Negotiation successful
0806 22:51:12.200479 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 22:51:12.203145 2871 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:51:12.203805 2871 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:51:12.208004 2871 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4016383b19624ad3a2221b7f665caacf: Generated new TSK 0
I20260806 22:51:12.208173 2871 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:51:12.213457 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 2766
2026-08-06T22:51:12Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1514 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056405724609-6305-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:51:12 dist-test-slave-zh2d krb5kdc[2903](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:51:12 dist-test-slave-zh2d krb5kdc[2903](info): set up 2 sockets
Aug 06 22:51:12 dist-test-slave-zh2d krb5kdc[2903](info): commencing operation
krb5kdc: starting...
W20260806 22:51:13.460295 6305 mini_kdc.cc:121] Time spent starting KDC: real 1.218s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056405724609-6305-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056405724609-6305-0/krb5kdc/test-user.keytab.
Aug 06 22:51:13 dist-test-slave-zh2d krb5kdc[2903](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056673, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:51:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:51:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056405724609-6305-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056405724609-6305-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 22:51:13.588567 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056405724609-6305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056405724609-6305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056405724609-6305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40671
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056405724609-6305-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40671
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:51:13.677937 2923 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:51:13.678191 2923 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:51:13.678232 2923 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:51:13.679410 2923 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:51:13.679486 2923 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:51:13.679513 2923 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:51:13.679548 2923 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:51:13.679581 2923 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:51:13.681206 2923 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056405724609-6305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056405724609-6305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40671
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056405724609-6305-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40671
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2923
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056405724609-6305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:51:13.681511 2923 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:51:13.681777 2923 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:51:13.687721 2931 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:51:13.687772 2935 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:51:13.687832 2923 server_base.cc:1034] running on GCE node
W20260806 22:51:13.687736 2930 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:51:13.688339 2923 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:51:13.688624 2923 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:51:13.689769 2923 hybrid_clock.cc:648] HybridClock initialized: now 1786056673689739 us; error 48 us; skew 500 ppm
Aug 06 22:51:13 dist-test-slave-zh2d krb5kdc[2903](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056673, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:51:13.697350 2923 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 22:51:13.697857 2923 webserver.cc:460] Webserver started at http://127.0.0.1:40533/ using document root <none> and password file <none>
I20260806 22:51:13.698115 2923 fs_manager.cc:359] Metadata directory not provided
I20260806 22:51:13.698186 2923 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:51:13.698334 2923 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:51:13.699429 2923 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056405724609-6305-0/minicluster-data/master-0/data/instance:
uuid: "8f585b3728a2422991b38d816be36470"
format_stamp: "Formatted at 2026-08-06 22:51:13 on dist-test-slave-zh2d"
server_key: "dbb17d94bfb51d8d234e43ed0f773db7"
server_key_iv: "11eebfd6bcfcd8303da0b4d7cb736a29"
server_key_version: "encryptionkey@0"
I20260806 22:51:13.699803 2923 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056405724609-6305-0/minicluster-data/master-0/wal/instance:
uuid: "8f585b3728a2422991b38d816be36470"
format_stamp: "Formatted at 2026-08-06 22:51:13 on dist-test-slave-zh2d"
server_key: "dbb17d94bfb51d8d234e43ed0f773db7"
server_key_iv: "11eebfd6bcfcd8303da0b4d7cb736a29"
server_key_version: "encryptionkey@0"
I20260806 22:51:13.701282 2923 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:51:13.702101 2944 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:51:13.702288 2923 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:51:13.702401 2923 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056405724609-6305-0/minicluster-data/master-0/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056405724609-6305-0/minicluster-data/master-0/wal
uuid: "8f585b3728a2422991b38d816be36470"
format_stamp: "Formatted at 2026-08-06 22:51:13 on dist-test-slave-zh2d"
server_key: "dbb17d94bfb51d8d234e43ed0f773db7"
server_key_iv: "11eebfd6bcfcd8303da0b4d7cb736a29"
server_key_version: "encryptionkey@0"
I20260806 22:51:13.702478 2923 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056405724609-6305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056405724609-6305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056405724609-6305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:51:13.718643 2923 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:51:13.720638 2923 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:51:13.720805 2923 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:51:13.725528 2923 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40671
I20260806 22:51:13.725580 3012 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40671 every 8 connection(s)
I20260806 22:51:13.725968 2923 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
I20260806 22:51:13.726997 3013 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:51:13.729400 3013 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f585b3728a2422991b38d816be36470: Bootstrap starting.
I20260806 22:51:13.729993 3013 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8f585b3728a2422991b38d816be36470: Neither blocks nor log segments found. Creating new log.
I20260806 22:51:13.730285 3013 log.cc:824] T 00000000000000000000000000000000 P 8f585b3728a2422991b38d816be36470: Log is configured to *not* fsync() on all Append() calls
I20260806 22:51:13.730988 3013 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f585b3728a2422991b38d816be36470: No bootstrap required, opened a new log
I20260806 22:51:13.732610 3013 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8f585b3728a2422991b38d816be36470 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f585b3728a2422991b38d816be36470" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40671 } }
I20260806 22:51:13.732754 3013 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8f585b3728a2422991b38d816be36470 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:51:13.732800 3013 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8f585b3728a2422991b38d816be36470 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f585b3728a2422991b38d816be36470, State: Initialized, Role: FOLLOWER
I20260806 22:51:13.732990 3013 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8f585b3728a2422991b38d816be36470 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f585b3728a2422991b38d816be36470" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40671 } }
I20260806 22:51:13.733103 3013 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8f585b3728a2422991b38d816be36470 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:51:13.733153 3013 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8f585b3728a2422991b38d816be36470 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:51:13.733204 3013 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8f585b3728a2422991b38d816be36470 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:51:13.733764 3013 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8f585b3728a2422991b38d816be36470 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f585b3728a2422991b38d816be36470" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40671 } }
I20260806 22:51:13.733851 3013 leader_election.cc:302] T 00000000000000000000000000000000 P 8f585b3728a2422991b38d816be36470 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f585b3728a2422991b38d816be36470; no voters:
I20260806 22:51:13.734040 3013 leader_election.cc:288] T 00000000000000000000000000000000 P 8f585b3728a2422991b38d816be36470 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:51:13.734074 3022 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8f585b3728a2422991b38d816be36470 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:51:13.734128 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 2923
I20260806 22:51:13.734248 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056405724609-6305-0/minicluster-data/master-0/wal/instance
I20260806 22:51:13.734272 3022 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8f585b3728a2422991b38d816be36470 [term 1 LEADER]: Becoming Leader. State: Replica: 8f585b3728a2422991b38d816be36470, State: Running, Role: LEADER
I20260806 22:51:13.734297 3013 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8f585b3728a2422991b38d816be36470 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:51:13.734359 3022 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8f585b3728a2422991b38d816be36470 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f585b3728a2422991b38d816be36470" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40671 } }
I20260806 22:51:13.734486 6305 external_mini_cluster.cc:1428] Setting key f19b57be959f37a7096469c7255d179d
I20260806 22:51:13.734699 3022 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f585b3728a2422991b38d816be36470 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8f585b3728a2422991b38d816be36470. Latest consensus state: current_term: 1 leader_uuid: "8f585b3728a2422991b38d816be36470" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f585b3728a2422991b38d816be36470" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40671 } } }
I20260806 22:51:13.734825 3022 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f585b3728a2422991b38d816be36470 [sys.catalog]: This master's current role is: LEADER
I20260806 22:51:13.735097 3039 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:51:13.735585 3027 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f585b3728a2422991b38d816be36470 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8f585b3728a2422991b38d816be36470" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f585b3728a2422991b38d816be36470" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40671 } } }
I20260806 22:51:13.735703 3027 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f585b3728a2422991b38d816be36470 [sys.catalog]: This master's current role is: LEADER
I20260806 22:51:13.736100 3039 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:51:13.737890 3039 catalog_manager.cc:1347] Generated new cluster ID: 10f093bef2a343ee8bb325f45792305f
I20260806 22:51:13.737946 3039 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:51:13.741099 3041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:13.735180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41107 (local address 127.0.0.1:40671)
0806 22:51:13.735603 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:13.735612 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:13.735643 (+ 31us) server_negotiation.cc:407] Connection header received
0806 22:51:13.735761 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:13.735768 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:13.735829 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:13.736361 (+ 532us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:13.736525 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:13.737772 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:13.738956 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:13.739553 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:13.739645 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:13.740392 (+ 747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:51:13.740396 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:51:13.740396 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:51:13.740598 (+ 202us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:51:13.740634 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:51:13.740647 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:51:13.740710 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:13.740823 (+ 113us) server_negotiation.cc:299] Negotiation successful
0806 22:51:13.740903 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":85,"threads_started":1}
I20260806 22:51:13.742017 6305 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:51:13.747043 3041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:13.743608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36131 (local address 127.0.0.1:40671)
0806 22:51:13.743646 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:13.743648 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:13.743814 (+ 166us) server_negotiation.cc:407] Connection header received
0806 22:51:13.743872 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:13.743873 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:13.743882 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:13.743947 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:13.744069 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:13.744751 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:13.745800 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:13.746099 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:13.746159 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:13.746836 (+ 677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:51:13.746837 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:51:13.746837 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:51:13.746841 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:51:13.746854 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:51:13.746858 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:51:13.746906 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:13.746953 (+ 47us) server_negotiation.cc:299] Negotiation successful
0806 22:51:13.746976 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 22:51:13.752025 3039 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:51:13.752733 3039 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:51:13.759027 3039 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8f585b3728a2422991b38d816be36470: Generated new TSK 0
I20260806 22:51:13.759192 3039 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:51:13.764562 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 2923
2026-08-06T22:51:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1551 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056405724609-6305-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:51:13 dist-test-slave-zh2d krb5kdc[3067](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:51:13 dist-test-slave-zh2d krb5kdc[3067](info): set up 2 sockets
Aug 06 22:51:13 dist-test-slave-zh2d krb5kdc[3067](info): commencing operation
krb5kdc: starting...
W20260806 22:51:15.021832 6305 mini_kdc.cc:121] Time spent starting KDC: real 1.229s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056405724609-6305-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056405724609-6305-0/krb5kdc/test-user.keytab.
Aug 06 22:51:15 dist-test-slave-zh2d krb5kdc[3067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056675, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:51:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:51:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056405724609-6305-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056405724609-6305-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 22:51:15.151919 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056405724609-6305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056405724609-6305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056405724609-6305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34907
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056405724609-6305-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34907
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:51:15.238824 3084 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:51:15.239085 3084 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:51:15.239125 3084 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:51:15.240302 3084 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:51:15.240384 3084 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:51:15.240408 3084 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:51:15.240428 3084 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:51:15.240460 3084 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:51:15.242004 3084 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056405724609-6305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056405724609-6305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34907
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056405724609-6305-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34907
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3084
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056405724609-6305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:51:15.242302 3084 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:51:15.242549 3084 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:51:15.247614 3093 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:51:15.247627 3094 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:51:15.247637 3096 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:51:15.247960 3084 server_base.cc:1034] running on GCE node
I20260806 22:51:15.248230 3084 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:51:15.248538 3084 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:51:15.249686 3084 hybrid_clock.cc:648] HybridClock initialized: now 1786056675249665 us; error 37 us; skew 500 ppm
Aug 06 22:51:15 dist-test-slave-zh2d krb5kdc[3067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056675, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:51:15.252267 3084 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 22:51:15.252758 3084 webserver.cc:460] Webserver started at http://127.0.0.1:36687/ using document root <none> and password file <none>
I20260806 22:51:15.253068 3084 fs_manager.cc:359] Metadata directory not provided
I20260806 22:51:15.253124 3084 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:51:15.253293 3084 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:51:15.254469 3084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056405724609-6305-0/minicluster-data/master-0/data/instance:
uuid: "6f80db8c8ec74eb484e8ba0d0217734a"
format_stamp: "Formatted at 2026-08-06 22:51:15 on dist-test-slave-zh2d"
server_key: "c97365b4ce74638fc3c281404b9e362d"
server_key_iv: "7d7502f249fab5f3693612514616cd07"
server_key_version: "encryptionkey@0"
I20260806 22:51:15.254863 3084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056405724609-6305-0/minicluster-data/master-0/wal/instance:
uuid: "6f80db8c8ec74eb484e8ba0d0217734a"
format_stamp: "Formatted at 2026-08-06 22:51:15 on dist-test-slave-zh2d"
server_key: "c97365b4ce74638fc3c281404b9e362d"
server_key_iv: "7d7502f249fab5f3693612514616cd07"
server_key_version: "encryptionkey@0"
I20260806 22:51:15.256289 3084 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:51:15.257155 3103 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:51:15.257345 3084 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:51:15.257457 3084 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056405724609-6305-0/minicluster-data/master-0/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056405724609-6305-0/minicluster-data/master-0/wal
uuid: "6f80db8c8ec74eb484e8ba0d0217734a"
format_stamp: "Formatted at 2026-08-06 22:51:15 on dist-test-slave-zh2d"
server_key: "c97365b4ce74638fc3c281404b9e362d"
server_key_iv: "7d7502f249fab5f3693612514616cd07"
server_key_version: "encryptionkey@0"
I20260806 22:51:15.257550 3084 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056405724609-6305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056405724609-6305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056405724609-6305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:51:15.261354 3084 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:51:15.263420 3084 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:51:15.263592 3084 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:51:15.268160 3084 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34907
I20260806 22:51:15.268204 3179 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34907 every 8 connection(s)
I20260806 22:51:15.268553 3084 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
I20260806 22:51:15.269649 3180 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:51:15.271903 3180 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f80db8c8ec74eb484e8ba0d0217734a: Bootstrap starting.
I20260806 22:51:15.272518 3180 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6f80db8c8ec74eb484e8ba0d0217734a: Neither blocks nor log segments found. Creating new log.
I20260806 22:51:15.272810 3180 log.cc:824] T 00000000000000000000000000000000 P 6f80db8c8ec74eb484e8ba0d0217734a: Log is configured to *not* fsync() on all Append() calls
I20260806 22:51:15.273581 3180 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f80db8c8ec74eb484e8ba0d0217734a: No bootstrap required, opened a new log
I20260806 22:51:15.275229 3180 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6f80db8c8ec74eb484e8ba0d0217734a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f80db8c8ec74eb484e8ba0d0217734a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34907 } }
I20260806 22:51:15.275363 3180 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6f80db8c8ec74eb484e8ba0d0217734a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:51:15.275386 3180 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6f80db8c8ec74eb484e8ba0d0217734a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f80db8c8ec74eb484e8ba0d0217734a, State: Initialized, Role: FOLLOWER
I20260806 22:51:15.275561 3180 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6f80db8c8ec74eb484e8ba0d0217734a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f80db8c8ec74eb484e8ba0d0217734a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34907 } }
I20260806 22:51:15.275640 3180 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6f80db8c8ec74eb484e8ba0d0217734a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:51:15.275691 3180 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6f80db8c8ec74eb484e8ba0d0217734a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:51:15.275758 3180 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6f80db8c8ec74eb484e8ba0d0217734a [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:51:15.276283 3180 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6f80db8c8ec74eb484e8ba0d0217734a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f80db8c8ec74eb484e8ba0d0217734a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34907 } }
I20260806 22:51:15.276398 3180 leader_election.cc:302] T 00000000000000000000000000000000 P 6f80db8c8ec74eb484e8ba0d0217734a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6f80db8c8ec74eb484e8ba0d0217734a; no voters:
I20260806 22:51:15.276584 3180 leader_election.cc:288] T 00000000000000000000000000000000 P 6f80db8c8ec74eb484e8ba0d0217734a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:51:15.276610 3185 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6f80db8c8ec74eb484e8ba0d0217734a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:51:15.276801 3185 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6f80db8c8ec74eb484e8ba0d0217734a [term 1 LEADER]: Becoming Leader. State: Replica: 6f80db8c8ec74eb484e8ba0d0217734a, State: Running, Role: LEADER
I20260806 22:51:15.276891 3180 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6f80db8c8ec74eb484e8ba0d0217734a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:51:15.276961 3185 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6f80db8c8ec74eb484e8ba0d0217734a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f80db8c8ec74eb484e8ba0d0217734a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34907 } }
I20260806 22:51:15.277526 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 3084
I20260806 22:51:15.277659 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056405724609-6305-0/minicluster-data/master-0/wal/instance
I20260806 22:51:15.277866 6305 external_mini_cluster.cc:1428] Setting key e3594f9ee45e49a5e9e8ab6a61b41c07
I20260806 22:51:15.277900 3186 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f80db8c8ec74eb484e8ba0d0217734a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6f80db8c8ec74eb484e8ba0d0217734a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f80db8c8ec74eb484e8ba0d0217734a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34907 } } }
I20260806 22:51:15.278012 3186 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f80db8c8ec74eb484e8ba0d0217734a [sys.catalog]: This master's current role is: LEADER
I20260806 22:51:15.278242 3206 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:51:15.278638 3187 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f80db8c8ec74eb484e8ba0d0217734a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6f80db8c8ec74eb484e8ba0d0217734a. Latest consensus state: current_term: 1 leader_uuid: "6f80db8c8ec74eb484e8ba0d0217734a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f80db8c8ec74eb484e8ba0d0217734a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34907 } } }
I20260806 22:51:15.278723 3187 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f80db8c8ec74eb484e8ba0d0217734a [sys.catalog]: This master's current role is: LEADER
I20260806 22:51:15.278967 3206 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:51:15.280462 3213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:15.278378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55011 (local address 127.0.0.1:34907)
0806 22:51:15.278914 (+ 536us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:15.278925 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:15.278982 (+ 57us) server_negotiation.cc:407] Connection header received
0806 22:51:15.279149 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:15.279154 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:15.279177 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:15.279323 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:15.280041 (+ 718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:51:15.280044 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:51:15.280045 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:51:15.280048 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:51:15.280091 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:51:15.280101 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:51:15.280174 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:15.280245 (+ 71us) server_negotiation.cc:299] Negotiation successful
0806 22:51:15.280287 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":480,"thread_start_us":62,"threads_started":1}
I20260806 22:51:15.281503 6305 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:51:15.283798 3213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:15.282813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55007 (local address 127.0.0.1:34907)
0806 22:51:15.282845 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:15.282847 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:15.282978 (+ 131us) server_negotiation.cc:407] Connection header received
0806 22:51:15.283033 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:15.283034 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:15.283038 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:15.283086 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:15.283611 (+ 525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:51:15.283612 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:51:15.283612 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:51:15.283616 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:51:15.283628 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:51:15.283632 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:51:15.283672 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:15.283736 (+ 64us) server_negotiation.cc:299] Negotiation successful
0806 22:51:15.283746 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260806 22:51:15.289263 3206 catalog_manager.cc:1347] Generated new cluster ID: 3005fdc9a9894f84a3cb36e55e194f13
I20260806 22:51:15.289342 3206 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:51:15.298743 3206 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:51:15.298936 3206 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:51:15.301194 3206 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6f80db8c8ec74eb484e8ba0d0217734a: Generated new TSK 0
I20260806 22:51:15.301378 3206 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:51:15.307152 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 3084
2026-08-06T22:51:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1546 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056405724609-6305-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:51:15 dist-test-slave-zh2d krb5kdc[3231](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:51:15 dist-test-slave-zh2d krb5kdc[3231](info): set up 2 sockets
Aug 06 22:51:15 dist-test-slave-zh2d krb5kdc[3231](info): commencing operation
krb5kdc: starting...
W20260806 22:51:16.537197 6305 mini_kdc.cc:121] Time spent starting KDC: real 1.197s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056405724609-6305-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056405724609-6305-0/krb5kdc/test-user.keytab.
Aug 06 22:51:16 dist-test-slave-zh2d krb5kdc[3231](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056676, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:51:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:51:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056405724609-6305-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056405724609-6305-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 22:51:16.659868 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056405724609-6305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056405724609-6305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056405724609-6305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36383
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056405724609-6305-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36383
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:51:16.743384 3252 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:51:16.743585 3252 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:51:16.743608 3252 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:51:16.744663 3252 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:51:16.744699 3252 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:51:16.744714 3252 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:51:16.744726 3252 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:51:16.744738 3252 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:51:16.746138 3252 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056405724609-6305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056405724609-6305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36383
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056405724609-6305-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36383
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3252
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056405724609-6305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:51:16.746323 3252 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:51:16.746495 3252 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:51:16.751596 3259 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:51:16.751735 3252 server_base.cc:1034] running on GCE node
W20260806 22:51:16.751614 3260 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:51:16.751595 3263 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:51:16.752087 3252 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:51:16.752331 3252 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:51:16.753537 3252 hybrid_clock.cc:648] HybridClock initialized: now 1786056676753518 us; error 50 us; skew 500 ppm
Aug 06 22:51:16 dist-test-slave-zh2d krb5kdc[3231](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056676, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:51:16.756488 3252 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 22:51:16.757058 3252 webserver.cc:460] Webserver started at http://127.0.0.1:43131/ using document root <none> and password file <none>
I20260806 22:51:16.757288 3252 fs_manager.cc:359] Metadata directory not provided
I20260806 22:51:16.757339 3252 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:51:16.757434 3252 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:51:16.758466 3252 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056405724609-6305-0/minicluster-data/master-0/data/instance:
uuid: "eb2f801813794b3fb7a9637285c5a1d7"
format_stamp: "Formatted at 2026-08-06 22:51:16 on dist-test-slave-zh2d"
server_key: "91e1f3a30263fdde9bc58ea4949a7828"
server_key_iv: "c66b9084b83b5ac2d1ca9b187d419794"
server_key_version: "encryptionkey@0"
I20260806 22:51:16.758803 3252 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056405724609-6305-0/minicluster-data/master-0/wal/instance:
uuid: "eb2f801813794b3fb7a9637285c5a1d7"
format_stamp: "Formatted at 2026-08-06 22:51:16 on dist-test-slave-zh2d"
server_key: "91e1f3a30263fdde9bc58ea4949a7828"
server_key_iv: "c66b9084b83b5ac2d1ca9b187d419794"
server_key_version: "encryptionkey@0"
I20260806 22:51:16.760180 3252 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:51:16.761008 3271 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:51:16.761186 3252 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:51:16.761274 3252 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056405724609-6305-0/minicluster-data/master-0/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056405724609-6305-0/minicluster-data/master-0/wal
uuid: "eb2f801813794b3fb7a9637285c5a1d7"
format_stamp: "Formatted at 2026-08-06 22:51:16 on dist-test-slave-zh2d"
server_key: "91e1f3a30263fdde9bc58ea4949a7828"
server_key_iv: "c66b9084b83b5ac2d1ca9b187d419794"
server_key_version: "encryptionkey@0"
I20260806 22:51:16.761329 3252 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056405724609-6305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056405724609-6305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056405724609-6305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:51:16.768651 3252 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:51:16.770519 3252 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:51:16.770625 3252 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:51:16.774610 3252 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36383
I20260806 22:51:16.774710 3347 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36383 every 8 connection(s)
I20260806 22:51:16.775022 3252 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
I20260806 22:51:16.775940 3348 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:51:16.778236 3348 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb2f801813794b3fb7a9637285c5a1d7: Bootstrap starting.
I20260806 22:51:16.778832 3348 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eb2f801813794b3fb7a9637285c5a1d7: Neither blocks nor log segments found. Creating new log.
I20260806 22:51:16.779135 3348 log.cc:824] T 00000000000000000000000000000000 P eb2f801813794b3fb7a9637285c5a1d7: Log is configured to *not* fsync() on all Append() calls
I20260806 22:51:16.779883 3348 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb2f801813794b3fb7a9637285c5a1d7: No bootstrap required, opened a new log
I20260806 22:51:16.781611 3348 raft_consensus.cc:348] T 00000000000000000000000000000000 P eb2f801813794b3fb7a9637285c5a1d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb2f801813794b3fb7a9637285c5a1d7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36383 } }
I20260806 22:51:16.781744 3348 raft_consensus.cc:374] T 00000000000000000000000000000000 P eb2f801813794b3fb7a9637285c5a1d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:51:16.781790 3348 raft_consensus.cc:729] T 00000000000000000000000000000000 P eb2f801813794b3fb7a9637285c5a1d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb2f801813794b3fb7a9637285c5a1d7, State: Initialized, Role: FOLLOWER
I20260806 22:51:16.781957 3348 consensus_queue.cc:260] T 00000000000000000000000000000000 P eb2f801813794b3fb7a9637285c5a1d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb2f801813794b3fb7a9637285c5a1d7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36383 } }
I20260806 22:51:16.782042 3348 raft_consensus.cc:388] T 00000000000000000000000000000000 P eb2f801813794b3fb7a9637285c5a1d7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:51:16.782094 3348 raft_consensus.cc:482] T 00000000000000000000000000000000 P eb2f801813794b3fb7a9637285c5a1d7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:51:16.782145 3348 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eb2f801813794b3fb7a9637285c5a1d7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:51:16.782694 3348 raft_consensus.cc:504] T 00000000000000000000000000000000 P eb2f801813794b3fb7a9637285c5a1d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb2f801813794b3fb7a9637285c5a1d7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36383 } }
I20260806 22:51:16.782814 3348 leader_election.cc:302] T 00000000000000000000000000000000 P eb2f801813794b3fb7a9637285c5a1d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eb2f801813794b3fb7a9637285c5a1d7; no voters:
I20260806 22:51:16.783010 3348 leader_election.cc:288] T 00000000000000000000000000000000 P eb2f801813794b3fb7a9637285c5a1d7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:51:16.783075 3355 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eb2f801813794b3fb7a9637285c5a1d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:51:16.783262 3348 sys_catalog.cc:564] T 00000000000000000000000000000000 P eb2f801813794b3fb7a9637285c5a1d7 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:51:16.783282 3355 raft_consensus.cc:686] T 00000000000000000000000000000000 P eb2f801813794b3fb7a9637285c5a1d7 [term 1 LEADER]: Becoming Leader. State: Replica: eb2f801813794b3fb7a9637285c5a1d7, State: Running, Role: LEADER
I20260806 22:51:16.783509 3355 consensus_queue.cc:237] T 00000000000000000000000000000000 P eb2f801813794b3fb7a9637285c5a1d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb2f801813794b3fb7a9637285c5a1d7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36383 } }
I20260806 22:51:16.783883 3357 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb2f801813794b3fb7a9637285c5a1d7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eb2f801813794b3fb7a9637285c5a1d7. Latest consensus state: current_term: 1 leader_uuid: "eb2f801813794b3fb7a9637285c5a1d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb2f801813794b3fb7a9637285c5a1d7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36383 } } }
I20260806 22:51:16.784011 3356 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb2f801813794b3fb7a9637285c5a1d7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eb2f801813794b3fb7a9637285c5a1d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb2f801813794b3fb7a9637285c5a1d7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36383 } } }
I20260806 22:51:16.784035 3357 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb2f801813794b3fb7a9637285c5a1d7 [sys.catalog]: This master's current role is: LEADER
I20260806 22:51:16.784103 3356 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb2f801813794b3fb7a9637285c5a1d7 [sys.catalog]: This master's current role is: LEADER
I20260806 22:51:16.784480 3371 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:51:16.785032 3371 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:51:16.785066 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 3252
I20260806 22:51:16.785172 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056405724609-6305-0/minicluster-data/master-0/wal/instance
I20260806 22:51:16.785418 6305 external_mini_cluster.cc:1428] Setting key bbcbd9892849d7f4b1efa48ebeb05202
I20260806 22:51:16.786681 3371 catalog_manager.cc:1347] Generated new cluster ID: 7b199fe00d9d4f7194597561f93bc543
I20260806 22:51:16.786751 3371 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 22:51:16 dist-test-slave-zh2d krb5kdc[3231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056676, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 22:51:16.799896 3386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:16.786178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.248:38089 (local address 127.0.0.1:36383)
0806 22:51:16.786360 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:16.786378 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:16.786431 (+ 53us) server_negotiation.cc:407] Connection header received
0806 22:51:16.786526 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:16.786531 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:16.786596 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:16.789000 (+ 2404us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:16.789769 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:16.791165 (+ 1396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:16.792366 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:16.792979 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:16.793105 (+ 126us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:16.795990 (+ 2885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:51:16.795994 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:51:16.795994 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:51:16.796007 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:51:16.798065 (+ 2058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:16.798493 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:16.798496 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:16.798496 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:16.798605 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:16.798845 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:16.798846 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:16.798846 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:16.799153 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:51:16.799295 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:16.799416 (+ 121us) server_negotiation.cc:299] Negotiation successful
0806 22:51:16.799637 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":72,"threads_started":1}
I20260806 22:51:16.800942 6305 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:51:16.803593 3371 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:51:16.804277 3371 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:51:16.808945 3386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:16.802443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.248:44445 (local address 127.0.0.1:36383)
0806 22:51:16.802492 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:16.802494 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:16.802649 (+ 155us) server_negotiation.cc:407] Connection header received
0806 22:51:16.802696 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:16.802697 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:16.802706 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:16.802770 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:16.803386 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:16.803982 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:16.804715 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:16.804993 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:16.805066 (+ 73us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:16.806285 (+ 1219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:51:16.806286 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:51:16.806287 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:51:16.806293 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:51:16.807731 (+ 1438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:16.808110 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:16.808111 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:16.808111 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:16.808166 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:16.808401 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:16.808402 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:16.808403 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:16.808531 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:51:16.808659 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:16.808770 (+ 111us) server_negotiation.cc:299] Negotiation successful
0806 22:51:16.808833 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 22:51:16.816555 3371 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eb2f801813794b3fb7a9637285c5a1d7: Generated new TSK 0
I20260806 22:51:16.816720 3371 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:51:16.822912 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 3252
W20260806 22:51:16.831837 3229 connection.cc:535] client connection to 127.0.0.1:36383 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36383): Connection reset by peer (error 104)
2026-08-06T22:51:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1505 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056405724609-6305-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:51:16 dist-test-slave-zh2d krb5kdc[3401](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:51:16 dist-test-slave-zh2d krb5kdc[3401](info): set up 2 sockets
Aug 06 22:51:16 dist-test-slave-zh2d krb5kdc[3401](info): commencing operation
krb5kdc: starting...
W20260806 22:51:18.141517 6305 mini_kdc.cc:121] Time spent starting KDC: real 1.296s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056405724609-6305-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056405724609-6305-0/krb5kdc/test-user.keytab.
Aug 06 22:51:18 dist-test-slave-zh2d krb5kdc[3401](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056678, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:51:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:51:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056405724609-6305-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056405724609-6305-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 22:51:18.267642 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056405724609-6305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056405724609-6305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056405724609-6305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38203
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056405724609-6305-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38203
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:51:18.371928 3423 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:51:18.372161 3423 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:51:18.372200 3423 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:51:18.374028 3423 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:51:18.374083 3423 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:51:18.374104 3423 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:51:18.374127 3423 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:51:18.374171 3423 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:51:18.375684 3423 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056405724609-6305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056405724609-6305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38203
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056405724609-6305-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38203
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3423
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056405724609-6305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:51:18.375941 3423 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:51:18.376195 3423 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 22:51:18.381796 3423 server_base.cc:1034] running on GCE node
W20260806 22:51:18.381724 3432 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:51:18.381731 3428 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:51:18.381723 3429 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:51:18.382467 3423 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:51:18.382793 3423 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:51:18.383991 3423 hybrid_clock.cc:648] HybridClock initialized: now 1786056678383986 us; error 42 us; skew 500 ppm
Aug 06 22:51:18 dist-test-slave-zh2d krb5kdc[3401](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056678, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:51:18.387135 3423 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 22:51:18.387661 3423 webserver.cc:460] Webserver started at http://127.0.0.1:33233/ using document root <none> and password file <none>
I20260806 22:51:18.387931 3423 fs_manager.cc:359] Metadata directory not provided
I20260806 22:51:18.388008 3423 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:51:18.388171 3423 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:51:18.389423 3423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056405724609-6305-0/minicluster-data/master-0/data/instance:
uuid: "e94965fce3a0404a9bba5f14da9daf9a"
format_stamp: "Formatted at 2026-08-06 22:51:18 on dist-test-slave-zh2d"
server_key: "b7248639ae83514bdc33ad1cd2a964b2"
server_key_iv: "e059216ac51e115fd2b637d1535016a2"
server_key_version: "encryptionkey@0"
I20260806 22:51:18.389818 3423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056405724609-6305-0/minicluster-data/master-0/wal/instance:
uuid: "e94965fce3a0404a9bba5f14da9daf9a"
format_stamp: "Formatted at 2026-08-06 22:51:18 on dist-test-slave-zh2d"
server_key: "b7248639ae83514bdc33ad1cd2a964b2"
server_key_iv: "e059216ac51e115fd2b637d1535016a2"
server_key_version: "encryptionkey@0"
I20260806 22:51:18.391270 3423 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:51:18.392100 3441 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:51:18.392297 3423 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:51:18.392424 3423 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056405724609-6305-0/minicluster-data/master-0/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056405724609-6305-0/minicluster-data/master-0/wal
uuid: "e94965fce3a0404a9bba5f14da9daf9a"
format_stamp: "Formatted at 2026-08-06 22:51:18 on dist-test-slave-zh2d"
server_key: "b7248639ae83514bdc33ad1cd2a964b2"
server_key_iv: "e059216ac51e115fd2b637d1535016a2"
server_key_version: "encryptionkey@0"
I20260806 22:51:18.392518 3423 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056405724609-6305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056405724609-6305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056405724609-6305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:51:18.404680 3423 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:51:18.406688 3423 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:51:18.406877 3423 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:51:18.411284 3423 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38203
I20260806 22:51:18.411336 3522 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38203 every 8 connection(s)
I20260806 22:51:18.411742 3423 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
I20260806 22:51:18.412737 3524 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:51:18.415107 3524 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e94965fce3a0404a9bba5f14da9daf9a: Bootstrap starting.
I20260806 22:51:18.415686 3524 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e94965fce3a0404a9bba5f14da9daf9a: Neither blocks nor log segments found. Creating new log.
I20260806 22:51:18.415972 3524 log.cc:824] T 00000000000000000000000000000000 P e94965fce3a0404a9bba5f14da9daf9a: Log is configured to *not* fsync() on all Append() calls
I20260806 22:51:18.416675 3524 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e94965fce3a0404a9bba5f14da9daf9a: No bootstrap required, opened a new log
I20260806 22:51:18.416950 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 3423
I20260806 22:51:18.417096 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056405724609-6305-0/minicluster-data/master-0/wal/instance
I20260806 22:51:18.417305 6305 external_mini_cluster.cc:1428] Setting key 9d0eac1384a97b61f6198736f8834e98
I20260806 22:51:18.419309 3524 raft_consensus.cc:348] T 00000000000000000000000000000000 P e94965fce3a0404a9bba5f14da9daf9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e94965fce3a0404a9bba5f14da9daf9a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38203 } }
I20260806 22:51:18.419520 3524 raft_consensus.cc:374] T 00000000000000000000000000000000 P e94965fce3a0404a9bba5f14da9daf9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:51:18.419556 3524 raft_consensus.cc:729] T 00000000000000000000000000000000 P e94965fce3a0404a9bba5f14da9daf9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e94965fce3a0404a9bba5f14da9daf9a, State: Initialized, Role: FOLLOWER
I20260806 22:51:18.419694 3524 consensus_queue.cc:260] T 00000000000000000000000000000000 P e94965fce3a0404a9bba5f14da9daf9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e94965fce3a0404a9bba5f14da9daf9a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38203 } }
I20260806 22:51:18.419788 3524 raft_consensus.cc:388] T 00000000000000000000000000000000 P e94965fce3a0404a9bba5f14da9daf9a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:51:18.419821 3524 raft_consensus.cc:482] T 00000000000000000000000000000000 P e94965fce3a0404a9bba5f14da9daf9a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:51:18.419868 3524 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e94965fce3a0404a9bba5f14da9daf9a [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:51:18.420552 3524 raft_consensus.cc:504] T 00000000000000000000000000000000 P e94965fce3a0404a9bba5f14da9daf9a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e94965fce3a0404a9bba5f14da9daf9a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38203 } }
I20260806 22:51:18.420670 3524 leader_election.cc:302] T 00000000000000000000000000000000 P e94965fce3a0404a9bba5f14da9daf9a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e94965fce3a0404a9bba5f14da9daf9a; no voters:
I20260806 22:51:18.420817 3524 leader_election.cc:288] T 00000000000000000000000000000000 P e94965fce3a0404a9bba5f14da9daf9a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:51:18.420917 3531 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e94965fce3a0404a9bba5f14da9daf9a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:51:18.421099 3531 raft_consensus.cc:686] T 00000000000000000000000000000000 P e94965fce3a0404a9bba5f14da9daf9a [term 1 LEADER]: Becoming Leader. State: Replica: e94965fce3a0404a9bba5f14da9daf9a, State: Running, Role: LEADER
I20260806 22:51:18.421133 3524 sys_catalog.cc:564] T 00000000000000000000000000000000 P e94965fce3a0404a9bba5f14da9daf9a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:51:18.422508 3531 consensus_queue.cc:237] T 00000000000000000000000000000000 P e94965fce3a0404a9bba5f14da9daf9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e94965fce3a0404a9bba5f14da9daf9a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38203 } }
I20260806 22:51:18.423116 3532 sys_catalog.cc:455] T 00000000000000000000000000000000 P e94965fce3a0404a9bba5f14da9daf9a [sys.catalog]: SysCatalogTable state changed. Reason: New leader e94965fce3a0404a9bba5f14da9daf9a. Latest consensus state: current_term: 1 leader_uuid: "e94965fce3a0404a9bba5f14da9daf9a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e94965fce3a0404a9bba5f14da9daf9a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38203 } } }
I20260806 22:51:18.423344 3532 sys_catalog.cc:458] T 00000000000000000000000000000000 P e94965fce3a0404a9bba5f14da9daf9a [sys.catalog]: This master's current role is: LEADER
I20260806 22:51:18.423162 3533 sys_catalog.cc:455] T 00000000000000000000000000000000 P e94965fce3a0404a9bba5f14da9daf9a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e94965fce3a0404a9bba5f14da9daf9a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e94965fce3a0404a9bba5f14da9daf9a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38203 } } }
I20260806 22:51:18.423492 3533 sys_catalog.cc:458] T 00000000000000000000000000000000 P e94965fce3a0404a9bba5f14da9daf9a [sys.catalog]: This master's current role is: LEADER
W20260806 22:51:18.423688 3546 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e94965fce3a0404a9bba5f14da9daf9a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:51:18.423770 3546 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:51:18.423955 3548 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:51:18.424191 3548 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:51:18.425005 3530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:18.417812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.248:34957 (local address 127.0.0.1:38203)
0806 22:51:18.418012 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:18.418021 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:18.418058 (+ 37us) server_negotiation.cc:407] Connection header received
0806 22:51:18.419504 (+ 1446us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:18.419514 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:18.419580 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:18.419736 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:18.420022 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:18.421426 (+ 1404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:18.422654 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:18.423309 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:18.423759 (+ 450us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:18.424246 (+ 487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:51:18.424249 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:51:18.424250 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:51:18.424457 (+ 207us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:51:18.424499 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:51:18.424515 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:51:18.424600 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:18.424744 (+ 144us) server_negotiation.cc:299] Negotiation successful
0806 22:51:18.424773 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":98,"threads_started":1}
I20260806 22:51:18.426065 6305 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:51:18.429251 3548 catalog_manager.cc:1347] Generated new cluster ID: 141e6527250a43d2b62989bf4366d23a
I20260806 22:51:18.429303 3548 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:51:18.430481 3530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:18.427311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.248:46771 (local address 127.0.0.1:38203)
0806 22:51:18.427348 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:18.427350 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:18.427482 (+ 132us) server_negotiation.cc:407] Connection header received
0806 22:51:18.427509 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:18.427510 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:18.427520 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:18.427585 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:18.427683 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:18.428385 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:18.429206 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:18.429469 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:18.429510 (+ 41us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:18.430228 (+ 718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:51:18.430229 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:51:18.430230 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:51:18.430234 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:51:18.430249 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:51:18.430254 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:51:18.430325 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:18.430402 (+ 77us) server_negotiation.cc:299] Negotiation successful
0806 22:51:18.430412 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 22:51:18.461869 3548 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:51:18.462453 3548 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:51:18.466174 3548 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e94965fce3a0404a9bba5f14da9daf9a: Generated new TSK 0
I20260806 22:51:18.466359 3548 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:51:18.474706 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 3423
W20260806 22:51:18.485476 3397 connection.cc:535] client connection to 127.0.0.1:38203 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38203): Connection reset by peer (error 104)
2026-08-06T22:51:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1653 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056405724609-6305-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:51:18 dist-test-slave-zh2d krb5kdc[3564](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:51:18 dist-test-slave-zh2d krb5kdc[3564](info): set up 2 sockets
Aug 06 22:51:18 dist-test-slave-zh2d krb5kdc[3564](info): commencing operation
krb5kdc: starting...
W20260806 22:51:19.683563 6305 mini_kdc.cc:121] Time spent starting KDC: real 1.184s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056405724609-6305-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056405724609-6305-0/krb5kdc/test-user.keytab.
Aug 06 22:51:19 dist-test-slave-zh2d krb5kdc[3564](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056679, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:51:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:51:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056405724609-6305-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056405724609-6305-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 22:51:19.811522 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056405724609-6305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056405724609-6305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056405724609-6305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39507
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056405724609-6305-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39507
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:51:19.898386 3581 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:51:19.898618 3581 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:51:19.898661 3581 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:51:19.899842 3581 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:51:19.899912 3581 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:51:19.899938 3581 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:51:19.899974 3581 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:51:19.900000 3581 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:51:19.901571 3581 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056405724609-6305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056405724609-6305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39507
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056405724609-6305-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39507
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3581
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056405724609-6305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:51:19.901862 3581 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:51:19.902101 3581 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:51:19.907326 3596 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:51:19.907626 3581 server_base.cc:1034] running on GCE node
W20260806 22:51:19.907343 3594 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:51:19.907325 3593 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:51:19.907995 3581 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:51:19.908258 3581 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:51:19.909407 3581 hybrid_clock.cc:648] HybridClock initialized: now 1786056679909377 us; error 49 us; skew 500 ppm
Aug 06 22:51:19 dist-test-slave-zh2d krb5kdc[3564](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056679, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:51:19.915982 3581 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 22:51:19.916481 3581 webserver.cc:460] Webserver started at http://127.0.0.1:41671/ using document root <none> and password file <none>
I20260806 22:51:19.916678 3581 fs_manager.cc:359] Metadata directory not provided
I20260806 22:51:19.916723 3581 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:51:19.916817 3581 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:51:19.917961 3581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056405724609-6305-0/minicluster-data/master-0/data/instance:
uuid: "82e1236cedf042c58de432783817403e"
format_stamp: "Formatted at 2026-08-06 22:51:19 on dist-test-slave-zh2d"
server_key: "8782d9e2c439666bb8c3b3ab9c94b3e5"
server_key_iv: "7edf7a2af3389f1d96ad7ee5e30710c6"
server_key_version: "encryptionkey@0"
I20260806 22:51:19.918341 3581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056405724609-6305-0/minicluster-data/master-0/wal/instance:
uuid: "82e1236cedf042c58de432783817403e"
format_stamp: "Formatted at 2026-08-06 22:51:19 on dist-test-slave-zh2d"
server_key: "8782d9e2c439666bb8c3b3ab9c94b3e5"
server_key_iv: "7edf7a2af3389f1d96ad7ee5e30710c6"
server_key_version: "encryptionkey@0"
I20260806 22:51:19.919756 3581 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:51:19.920542 3604 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:51:19.920760 3581 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:51:19.920876 3581 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056405724609-6305-0/minicluster-data/master-0/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056405724609-6305-0/minicluster-data/master-0/wal
uuid: "82e1236cedf042c58de432783817403e"
format_stamp: "Formatted at 2026-08-06 22:51:19 on dist-test-slave-zh2d"
server_key: "8782d9e2c439666bb8c3b3ab9c94b3e5"
server_key_iv: "7edf7a2af3389f1d96ad7ee5e30710c6"
server_key_version: "encryptionkey@0"
I20260806 22:51:19.920958 3581 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056405724609-6305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056405724609-6305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056405724609-6305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:51:19.924319 3581 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:51:19.926582 3581 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:51:19.926767 3581 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:51:19.931550 3581 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39507
I20260806 22:51:19.931581 3672 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39507 every 8 connection(s)
I20260806 22:51:19.931936 3581 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
I20260806 22:51:19.932981 3673 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:51:19.935266 3673 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82e1236cedf042c58de432783817403e: Bootstrap starting.
I20260806 22:51:19.935847 3673 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 82e1236cedf042c58de432783817403e: Neither blocks nor log segments found. Creating new log.
I20260806 22:51:19.936158 3673 log.cc:824] T 00000000000000000000000000000000 P 82e1236cedf042c58de432783817403e: Log is configured to *not* fsync() on all Append() calls
I20260806 22:51:19.936825 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 3581
I20260806 22:51:19.936939 3673 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82e1236cedf042c58de432783817403e: No bootstrap required, opened a new log
I20260806 22:51:19.937009 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056405724609-6305-0/minicluster-data/master-0/wal/instance
I20260806 22:51:19.937193 6305 external_mini_cluster.cc:1428] Setting key ada8f3c8ee134c4192e99981b6be99cf
I20260806 22:51:19.939399 3673 raft_consensus.cc:348] T 00000000000000000000000000000000 P 82e1236cedf042c58de432783817403e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82e1236cedf042c58de432783817403e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39507 } }
I20260806 22:51:19.939602 3673 raft_consensus.cc:374] T 00000000000000000000000000000000 P 82e1236cedf042c58de432783817403e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:51:19.939653 3673 raft_consensus.cc:729] T 00000000000000000000000000000000 P 82e1236cedf042c58de432783817403e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82e1236cedf042c58de432783817403e, State: Initialized, Role: FOLLOWER
I20260806 22:51:19.939776 3673 consensus_queue.cc:260] T 00000000000000000000000000000000 P 82e1236cedf042c58de432783817403e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82e1236cedf042c58de432783817403e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39507 } }
I20260806 22:51:19.939857 3683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:19.937742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.248:40101 (local address 127.0.0.1:39507)
0806 22:51:19.937938 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:19.937947 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:19.937981 (+ 34us) server_negotiation.cc:407] Connection header received
0806 22:51:19.938463 (+ 482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:19.938472 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:19.938505 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:19.938655 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:19.939260 (+ 605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:51:19.939264 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:51:19.939265 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:51:19.939269 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:51:19.939313 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:51:19.939327 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:51:19.939405 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:19.939466 (+ 61us) server_negotiation.cc:299] Negotiation successful
0806 22:51:19.939508 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":91,"threads_started":1}
I20260806 22:51:19.939880 3673 raft_consensus.cc:388] T 00000000000000000000000000000000 P 82e1236cedf042c58de432783817403e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:51:19.939920 3673 raft_consensus.cc:482] T 00000000000000000000000000000000 P 82e1236cedf042c58de432783817403e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:51:19.939976 3673 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 82e1236cedf042c58de432783817403e [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:51:19.940713 3673 raft_consensus.cc:504] T 00000000000000000000000000000000 P 82e1236cedf042c58de432783817403e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82e1236cedf042c58de432783817403e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39507 } }
I20260806 22:51:19.940827 3673 leader_election.cc:302] T 00000000000000000000000000000000 P 82e1236cedf042c58de432783817403e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 82e1236cedf042c58de432783817403e; no voters:
I20260806 22:51:19.940828 6305 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:51:19.941030 3673 leader_election.cc:288] T 00000000000000000000000000000000 P 82e1236cedf042c58de432783817403e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:51:19.941054 3685 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 82e1236cedf042c58de432783817403e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:51:19.941169 3685 raft_consensus.cc:686] T 00000000000000000000000000000000 P 82e1236cedf042c58de432783817403e [term 1 LEADER]: Becoming Leader. State: Replica: 82e1236cedf042c58de432783817403e, State: Running, Role: LEADER
I20260806 22:51:19.941280 3673 sys_catalog.cc:564] T 00000000000000000000000000000000 P 82e1236cedf042c58de432783817403e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:51:19.941303 3685 consensus_queue.cc:237] T 00000000000000000000000000000000 P 82e1236cedf042c58de432783817403e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82e1236cedf042c58de432783817403e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39507 } }
I20260806 22:51:19.941668 3685 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82e1236cedf042c58de432783817403e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 82e1236cedf042c58de432783817403e. Latest consensus state: current_term: 1 leader_uuid: "82e1236cedf042c58de432783817403e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82e1236cedf042c58de432783817403e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39507 } } }
I20260806 22:51:19.941681 3686 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82e1236cedf042c58de432783817403e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "82e1236cedf042c58de432783817403e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82e1236cedf042c58de432783817403e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39507 } } }
I20260806 22:51:19.941849 3685 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82e1236cedf042c58de432783817403e [sys.catalog]: This master's current role is: LEADER
I20260806 22:51:19.941859 3686 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82e1236cedf042c58de432783817403e [sys.catalog]: This master's current role is: LEADER
I20260806 22:51:19.942181 3706 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:51:19.942947 3706 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:51:19.944223 3683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:19.942858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.248:43813 (local address 127.0.0.1:39507)
0806 22:51:19.942893 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:19.942895 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:19.943287 (+ 392us) server_negotiation.cc:407] Connection header received
0806 22:51:19.943337 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:19.943338 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:19.943342 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:19.943403 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:19.943993 (+ 590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:51:19.943994 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:51:19.943995 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:51:19.943999 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:51:19.944014 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:51:19.944019 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:51:19.944076 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:19.944144 (+ 68us) server_negotiation.cc:299] Negotiation successful
0806 22:51:19.944156 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 22:51:19.944932 3706 catalog_manager.cc:1347] Generated new cluster ID: 7691549d68e343908f227a7ae0638275
I20260806 22:51:19.944983 3706 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:51:19.955479 3706 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:51:19.955654 3706 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:51:19.958738 3706 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 82e1236cedf042c58de432783817403e: Generated new TSK 0
I20260806 22:51:19.958894 3706 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:51:19.965082 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 3581
2026-08-06T22:51:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1493 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056405724609-6305-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:51:20 dist-test-slave-zh2d krb5kdc[3725](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:51:20 dist-test-slave-zh2d krb5kdc[3725](info): set up 2 sockets
Aug 06 22:51:20 dist-test-slave-zh2d krb5kdc[3725](info): commencing operation
krb5kdc: starting...
W20260806 22:51:21.174805 6305 mini_kdc.cc:121] Time spent starting KDC: real 1.182s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056405724609-6305-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056405724609-6305-0/krb5kdc/test-user.keytab.
Aug 06 22:51:21 dist-test-slave-zh2d krb5kdc[3725](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056681, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:51:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:51:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056405724609-6305-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056405724609-6305-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 22:51:21.303352 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056405724609-6305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056405724609-6305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056405724609-6305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40545
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056405724609-6305-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40545
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:51:21.387275 3745 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:51:21.387502 3745 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:51:21.387542 3745 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:51:21.388674 3745 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:51:21.388742 3745 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:51:21.388767 3745 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:51:21.388795 3745 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:51:21.388826 3745 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:51:21.390352 3745 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056405724609-6305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056405724609-6305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40545
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056405724609-6305-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40545
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3745
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056405724609-6305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:51:21.390635 3745 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:51:21.390861 3745 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:51:21.395990 3753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:51:21.395998 3755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:51:21.396054 3752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:51:21.396378 3745 server_base.cc:1034] running on GCE node
I20260806 22:51:21.396639 3745 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:51:21.396992 3745 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:51:21.398162 3745 hybrid_clock.cc:648] HybridClock initialized: now 1786056681398159 us; error 48 us; skew 500 ppm
Aug 06 22:51:21 dist-test-slave-zh2d krb5kdc[3725](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056681, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:51:21.400651 3745 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 22:51:21.401191 3745 webserver.cc:460] Webserver started at http://127.0.0.1:42955/ using document root <none> and password file <none>
I20260806 22:51:21.401449 3745 fs_manager.cc:359] Metadata directory not provided
I20260806 22:51:21.401515 3745 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:51:21.401669 3745 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:51:21.402712 3745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056405724609-6305-0/minicluster-data/master-0/data/instance:
uuid: "16c9a711b0144d3dad9ea1e367d2e65a"
format_stamp: "Formatted at 2026-08-06 22:51:21 on dist-test-slave-zh2d"
server_key: "56763d724fcd851495a559b59eb35f52"
server_key_iv: "8bc94135569c4cc2f12210a64dae75a3"
server_key_version: "encryptionkey@0"
I20260806 22:51:21.403074 3745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056405724609-6305-0/minicluster-data/master-0/wal/instance:
uuid: "16c9a711b0144d3dad9ea1e367d2e65a"
format_stamp: "Formatted at 2026-08-06 22:51:21 on dist-test-slave-zh2d"
server_key: "56763d724fcd851495a559b59eb35f52"
server_key_iv: "8bc94135569c4cc2f12210a64dae75a3"
server_key_version: "encryptionkey@0"
I20260806 22:51:21.404474 3745 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 22:51:21.405313 3764 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:51:21.405494 3745 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 22:51:21.405593 3745 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056405724609-6305-0/minicluster-data/master-0/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056405724609-6305-0/minicluster-data/master-0/wal
uuid: "16c9a711b0144d3dad9ea1e367d2e65a"
format_stamp: "Formatted at 2026-08-06 22:51:21 on dist-test-slave-zh2d"
server_key: "56763d724fcd851495a559b59eb35f52"
server_key_iv: "8bc94135569c4cc2f12210a64dae75a3"
server_key_version: "encryptionkey@0"
I20260806 22:51:21.405673 3745 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056405724609-6305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056405724609-6305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056405724609-6305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:51:21.426906 3745 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:51:21.428778 3745 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:51:21.428977 3745 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:51:21.433432 3745 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40545
I20260806 22:51:21.433485 3840 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40545 every 8 connection(s)
I20260806 22:51:21.433856 3745 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
I20260806 22:51:21.434819 3844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:51:21.437166 3844 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16c9a711b0144d3dad9ea1e367d2e65a: Bootstrap starting.
I20260806 22:51:21.437762 3844 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 16c9a711b0144d3dad9ea1e367d2e65a: Neither blocks nor log segments found. Creating new log.
I20260806 22:51:21.438055 3844 log.cc:824] T 00000000000000000000000000000000 P 16c9a711b0144d3dad9ea1e367d2e65a: Log is configured to *not* fsync() on all Append() calls
I20260806 22:51:21.438803 3844 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16c9a711b0144d3dad9ea1e367d2e65a: No bootstrap required, opened a new log
I20260806 22:51:21.438792 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 3745
I20260806 22:51:21.438920 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056405724609-6305-0/minicluster-data/master-0/wal/instance
I20260806 22:51:21.439145 6305 external_mini_cluster.cc:1428] Setting key 7c5c175865e7af3ebf8f739fb4997578
I20260806 22:51:21.440791 3844 raft_consensus.cc:348] T 00000000000000000000000000000000 P 16c9a711b0144d3dad9ea1e367d2e65a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16c9a711b0144d3dad9ea1e367d2e65a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40545 } }
I20260806 22:51:21.441028 3844 raft_consensus.cc:374] T 00000000000000000000000000000000 P 16c9a711b0144d3dad9ea1e367d2e65a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:51:21.441114 3844 raft_consensus.cc:729] T 00000000000000000000000000000000 P 16c9a711b0144d3dad9ea1e367d2e65a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16c9a711b0144d3dad9ea1e367d2e65a, State: Initialized, Role: FOLLOWER
I20260806 22:51:21.441256 3844 consensus_queue.cc:260] T 00000000000000000000000000000000 P 16c9a711b0144d3dad9ea1e367d2e65a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16c9a711b0144d3dad9ea1e367d2e65a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40545 } }
I20260806 22:51:21.441360 3844 raft_consensus.cc:388] T 00000000000000000000000000000000 P 16c9a711b0144d3dad9ea1e367d2e65a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:51:21.441406 3844 raft_consensus.cc:482] T 00000000000000000000000000000000 P 16c9a711b0144d3dad9ea1e367d2e65a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:51:21.441460 3844 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 16c9a711b0144d3dad9ea1e367d2e65a [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:51:21.442427 3844 raft_consensus.cc:504] T 00000000000000000000000000000000 P 16c9a711b0144d3dad9ea1e367d2e65a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16c9a711b0144d3dad9ea1e367d2e65a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40545 } }
I20260806 22:51:21.442518 3844 leader_election.cc:302] T 00000000000000000000000000000000 P 16c9a711b0144d3dad9ea1e367d2e65a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 16c9a711b0144d3dad9ea1e367d2e65a; no voters:
I20260806 22:51:21.442699 3844 leader_election.cc:288] T 00000000000000000000000000000000 P 16c9a711b0144d3dad9ea1e367d2e65a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:51:21.442759 3849 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 16c9a711b0144d3dad9ea1e367d2e65a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:51:21.442895 3849 raft_consensus.cc:686] T 00000000000000000000000000000000 P 16c9a711b0144d3dad9ea1e367d2e65a [term 1 LEADER]: Becoming Leader. State: Replica: 16c9a711b0144d3dad9ea1e367d2e65a, State: Running, Role: LEADER
I20260806 22:51:21.442961 3844 sys_catalog.cc:564] T 00000000000000000000000000000000 P 16c9a711b0144d3dad9ea1e367d2e65a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:51:21.443078 3849 consensus_queue.cc:237] T 00000000000000000000000000000000 P 16c9a711b0144d3dad9ea1e367d2e65a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16c9a711b0144d3dad9ea1e367d2e65a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40545 } }
I20260806 22:51:21.444341 3850 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16c9a711b0144d3dad9ea1e367d2e65a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "16c9a711b0144d3dad9ea1e367d2e65a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16c9a711b0144d3dad9ea1e367d2e65a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40545 } } }
I20260806 22:51:21.444444 3850 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16c9a711b0144d3dad9ea1e367d2e65a [sys.catalog]: This master's current role is: LEADER
I20260806 22:51:21.444949 3851 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16c9a711b0144d3dad9ea1e367d2e65a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 16c9a711b0144d3dad9ea1e367d2e65a. Latest consensus state: current_term: 1 leader_uuid: "16c9a711b0144d3dad9ea1e367d2e65a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16c9a711b0144d3dad9ea1e367d2e65a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40545 } } }
W20260806 22:51:21.444965 3872 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 16c9a711b0144d3dad9ea1e367d2e65a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:51:21.445318 3872 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:51:21.445370 3873 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:51:21.445129 3851 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16c9a711b0144d3dad9ea1e367d2e65a [sys.catalog]: This master's current role is: LEADER
I20260806 22:51:21.445552 3873 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 06 22:51:21 dist-test-slave-zh2d krb5kdc[3725](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056681, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 22:51:21.447075 3873 catalog_manager.cc:1347] Generated new cluster ID: 87201ed37d304c1dac5cce0e28c53fe8
I20260806 22:51:21.447135 3873 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:51:21.451263 3848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:21.439687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.248:36113 (local address 127.0.0.1:40545)
0806 22:51:21.439849 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:21.439857 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:21.439895 (+ 38us) server_negotiation.cc:407] Connection header received
0806 22:51:21.440775 (+ 880us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:21.440780 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:21.440828 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:21.440993 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:21.441826 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:21.442912 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:21.444116 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:21.444775 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:21.445158 (+ 383us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:21.447432 (+ 2274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:51:21.447436 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:51:21.447436 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:51:21.447456 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:51:21.449475 (+ 2019us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:21.449936 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:21.449941 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:21.449941 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:21.450089 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:21.450331 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:21.450332 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:21.450332 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:21.450659 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:51:21.450830 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:21.450926 (+ 96us) server_negotiation.cc:299] Negotiation successful
0806 22:51:21.451038 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":77,"threads_started":1}
I20260806 22:51:21.452440 6305 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:51:21.458433 3873 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:51:21.459197 3873 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:51:21.461499 3848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:21.454041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.248:35901 (local address 127.0.0.1:40545)
0806 22:51:21.454080 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:21.454082 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:21.454244 (+ 162us) server_negotiation.cc:407] Connection header received
0806 22:51:21.454300 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:21.454301 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:21.454312 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:21.454364 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:21.454989 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:21.455724 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:21.456748 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:21.457160 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:21.457229 (+ 69us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:21.458634 (+ 1405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:51:21.458636 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:51:21.458636 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:51:21.458643 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:51:21.460248 (+ 1605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:21.460672 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:21.460673 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:21.460674 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:21.460727 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:21.460986 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:21.460986 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:21.460987 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:21.461105 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:51:21.461192 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:21.461357 (+ 165us) server_negotiation.cc:299] Negotiation successful
0806 22:51:21.461416 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 22:51:21.468825 3873 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 16c9a711b0144d3dad9ea1e367d2e65a: Generated new TSK 0
I20260806 22:51:21.469024 3873 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:51:21.472496 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 3745
W20260806 22:51:21.487155 3723 connection.cc:535] client connection to 127.0.0.1:40545 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40545): Connection reset by peer (error 104)
2026-08-06T22:51:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1507 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056405724609-6305-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:51:21 dist-test-slave-zh2d krb5kdc[3888](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:51:21 dist-test-slave-zh2d krb5kdc[3888](info): set up 2 sockets
Aug 06 22:51:21 dist-test-slave-zh2d krb5kdc[3888](info): commencing operation
krb5kdc: starting...
W20260806 22:51:22.712469 6305 mini_kdc.cc:121] Time spent starting KDC: real 1.211s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056405724609-6305-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056405724609-6305-0/krb5kdc/test-user.keytab.
Aug 06 22:51:22 dist-test-slave-zh2d krb5kdc[3888](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056682, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:51:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:51:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056405724609-6305-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056405724609-6305-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 22:51:22.839442 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056405724609-6305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056405724609-6305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056405724609-6305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42335
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056405724609-6305-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42057
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42335
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:51:22.922355 3909 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:51:22.922566 3909 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:51:22.922590 3909 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:51:22.923658 3909 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:51:22.923692 3909 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:51:22.923707 3909 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:51:22.923719 3909 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:51:22.923731 3909 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:51:22.925137 3909 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42057
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056405724609-6305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056405724609-6305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42335
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056405724609-6305-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42335
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3909
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056405724609-6305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:51:22.925321 3909 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:51:22.925491 3909 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:51:22.930353 3919 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:51:22.930352 3915 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:51:22.930387 3917 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:51:22.930711 3909 server_base.cc:1034] running on GCE node
I20260806 22:51:22.930976 3909 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:51:22.931319 3909 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:51:22.932497 3909 hybrid_clock.cc:648] HybridClock initialized: now 1786056682932489 us; error 35 us; skew 500 ppm
Aug 06 22:51:22 dist-test-slave-zh2d krb5kdc[3888](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056682, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:51:22.935185 3909 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 22:51:22.935599 3909 webserver.cc:460] Webserver started at http://127.0.0.1:35271/ using document root <none> and password file <none>
I20260806 22:51:22.935835 3909 fs_manager.cc:359] Metadata directory not provided
I20260806 22:51:22.935880 3909 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:51:22.936030 3909 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:51:22.937134 3909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056405724609-6305-0/minicluster-data/master-0/data/instance:
uuid: "3255929c286441149d0d5ac832cbf51d"
format_stamp: "Formatted at 2026-08-06 22:51:22 on dist-test-slave-zh2d"
server_key: "fefd604b44d0d0c90d4b4c82f5d1e440"
server_key_iv: "e48a868655e05256877f43a0f5733d0d"
server_key_version: "encryptionkey@0"
I20260806 22:51:22.937495 3909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056405724609-6305-0/minicluster-data/master-0/wal/instance:
uuid: "3255929c286441149d0d5ac832cbf51d"
format_stamp: "Formatted at 2026-08-06 22:51:22 on dist-test-slave-zh2d"
server_key: "fefd604b44d0d0c90d4b4c82f5d1e440"
server_key_iv: "e48a868655e05256877f43a0f5733d0d"
server_key_version: "encryptionkey@0"
I20260806 22:51:22.938812 3909 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260806 22:51:22.939499 3928 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:51:22.939662 3909 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 22:51:22.939767 3909 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056405724609-6305-0/minicluster-data/master-0/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056405724609-6305-0/minicluster-data/master-0/wal
uuid: "3255929c286441149d0d5ac832cbf51d"
format_stamp: "Formatted at 2026-08-06 22:51:22 on dist-test-slave-zh2d"
server_key: "fefd604b44d0d0c90d4b4c82f5d1e440"
server_key_iv: "e48a868655e05256877f43a0f5733d0d"
server_key_version: "encryptionkey@0"
I20260806 22:51:22.939839 3909 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056405724609-6305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056405724609-6305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056405724609-6305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:51:22.970306 3909 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:51:22.972783 3909 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:51:22.972980 3909 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:51:22.977535 3909 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42335
I20260806 22:51:22.977605 4003 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42335 every 8 connection(s)
I20260806 22:51:22.977977 3909 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
I20260806 22:51:22.978924 4004 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:51:22.981508 4004 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3255929c286441149d0d5ac832cbf51d: Bootstrap starting.
I20260806 22:51:22.982159 4004 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3255929c286441149d0d5ac832cbf51d: Neither blocks nor log segments found. Creating new log.
I20260806 22:51:22.982484 4004 log.cc:824] T 00000000000000000000000000000000 P 3255929c286441149d0d5ac832cbf51d: Log is configured to *not* fsync() on all Append() calls
I20260806 22:51:22.983223 4004 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3255929c286441149d0d5ac832cbf51d: No bootstrap required, opened a new log
I20260806 22:51:22.984959 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 3909
I20260806 22:51:22.985100 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056405724609-6305-0/minicluster-data/master-0/wal/instance
I20260806 22:51:22.985328 6305 external_mini_cluster.cc:1428] Setting key d4d74a616efafae3276166a8dffbce6a
I20260806 22:51:22.985265 4004 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3255929c286441149d0d5ac832cbf51d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3255929c286441149d0d5ac832cbf51d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42335 } }
I20260806 22:51:22.985472 4004 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3255929c286441149d0d5ac832cbf51d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:51:22.985522 4004 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3255929c286441149d0d5ac832cbf51d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3255929c286441149d0d5ac832cbf51d, State: Initialized, Role: FOLLOWER
I20260806 22:51:22.985730 4004 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3255929c286441149d0d5ac832cbf51d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3255929c286441149d0d5ac832cbf51d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42335 } }
I20260806 22:51:22.985846 4004 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3255929c286441149d0d5ac832cbf51d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:51:22.985965 4004 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3255929c286441149d0d5ac832cbf51d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:51:22.986047 4004 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3255929c286441149d0d5ac832cbf51d [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:51:22.987372 4004 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3255929c286441149d0d5ac832cbf51d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3255929c286441149d0d5ac832cbf51d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42335 } }
I20260806 22:51:22.987504 4004 leader_election.cc:302] T 00000000000000000000000000000000 P 3255929c286441149d0d5ac832cbf51d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3255929c286441149d0d5ac832cbf51d; no voters:
I20260806 22:51:22.987676 4004 leader_election.cc:288] T 00000000000000000000000000000000 P 3255929c286441149d0d5ac832cbf51d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:51:22.987735 4012 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3255929c286441149d0d5ac832cbf51d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:51:22.987910 4012 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3255929c286441149d0d5ac832cbf51d [term 1 LEADER]: Becoming Leader. State: Replica: 3255929c286441149d0d5ac832cbf51d, State: Running, Role: LEADER
I20260806 22:51:22.987933 4004 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3255929c286441149d0d5ac832cbf51d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:51:22.988061 4012 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3255929c286441149d0d5ac832cbf51d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3255929c286441149d0d5ac832cbf51d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42335 } }
I20260806 22:51:22.988497 4016 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3255929c286441149d0d5ac832cbf51d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3255929c286441149d0d5ac832cbf51d. Latest consensus state: current_term: 1 leader_uuid: "3255929c286441149d0d5ac832cbf51d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3255929c286441149d0d5ac832cbf51d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42335 } } }
I20260806 22:51:22.988631 4016 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3255929c286441149d0d5ac832cbf51d [sys.catalog]: This master's current role is: LEADER
I20260806 22:51:22.988912 4023 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:51:22.989457 4014 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3255929c286441149d0d5ac832cbf51d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3255929c286441149d0d5ac832cbf51d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3255929c286441149d0d5ac832cbf51d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42335 } } }
I20260806 22:51:22.989554 4014 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3255929c286441149d0d5ac832cbf51d [sys.catalog]: This master's current role is: LEADER
I20260806 22:51:22.989660 4023 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:51:22.991312 4023 catalog_manager.cc:1347] Generated new cluster ID: f3294d7a566940be833356a8ca43d5ba
I20260806 22:51:22.991380 4023 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:51:22.991883 4011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:22.985891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.248:33389 (local address 127.0.0.1:42335)
0806 22:51:22.986146 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:22.986157 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:22.986195 (+ 38us) server_negotiation.cc:407] Connection header received
0806 22:51:22.986332 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:22.986340 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:22.986407 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:22.986598 (+ 191us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:22.986774 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:22.988342 (+ 1568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:22.989577 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:22.990150 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:22.990264 (+ 114us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:22.991093 (+ 829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:51:22.991098 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:51:22.991098 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:51:22.991321 (+ 223us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:51:22.991361 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:51:22.991378 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:51:22.991475 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:22.991651 (+ 176us) server_negotiation.cc:299] Negotiation successful
0806 22:51:22.991679 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":119,"threads_started":1}
I20260806 22:51:22.993001 6305 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:51:22.998682 4011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:22.994446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.248:52539 (local address 127.0.0.1:42335)
0806 22:51:22.994488 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:22.994490 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:22.994632 (+ 142us) server_negotiation.cc:407] Connection header received
0806 22:51:22.994664 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:22.994665 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:22.994677 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:22.994756 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:22.994856 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:22.995641 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:22.996640 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:22.997477 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:22.997566 (+ 89us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:22.998337 (+ 771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:51:22.998338 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:51:22.998338 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:51:22.998343 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:51:22.998360 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:51:22.998364 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:51:22.998433 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:22.998583 (+ 150us) server_negotiation.cc:299] Negotiation successful
0806 22:51:22.998599 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 22:51:22.999207 4023 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:51:22.999698 4023 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:51:23.005815 4023 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3255929c286441149d0d5ac832cbf51d: Generated new TSK 0
I20260806 22:51:23.006021 4023 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:51:23.013767 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 3909
W20260806 22:51:23.030651 3884 connection.cc:535] client connection to 127.0.0.1:42335 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42335): Connection reset by peer (error 104)
2026-08-06T22:51:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1543 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056405724609-6305-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:51:23 dist-test-slave-zh2d krb5kdc[4047](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:51:23 dist-test-slave-zh2d krb5kdc[4047](info): set up 2 sockets
Aug 06 22:51:23 dist-test-slave-zh2d krb5kdc[4047](info): commencing operation
krb5kdc: starting...
W20260806 22:51:24.224699 6305 mini_kdc.cc:121] Time spent starting KDC: real 1.180s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056405724609-6305-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056405724609-6305-0/krb5kdc/test-user.keytab.
Aug 06 22:51:24 dist-test-slave-zh2d krb5kdc[4047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056684, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:51:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:51:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056405724609-6305-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056405724609-6305-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 22:51:24.351735 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056405724609-6305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056405724609-6305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056405724609-6305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44505
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056405724609-6305-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44505
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:51:24.436581 4062 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:51:24.436825 4062 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:51:24.436879 4062 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:51:24.437979 4062 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:51:24.438014 4062 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:51:24.438028 4062 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:51:24.438040 4062 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:51:24.438052 4062 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:51:24.439519 4062 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056405724609-6305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056405724609-6305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44505
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056405724609-6305-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44505
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4062
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056405724609-6305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:51:24.439709 4062 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:51:24.439880 4062 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:51:24.444823 4070 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:51:24.444895 4072 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:51:24.444928 4062 server_base.cc:1034] running on GCE node
W20260806 22:51:24.444839 4068 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:51:24.445266 4062 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:51:24.445590 4062 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:51:24.446766 4062 hybrid_clock.cc:648] HybridClock initialized: now 1786056684446749 us; error 38 us; skew 500 ppm
Aug 06 22:51:24 dist-test-slave-zh2d krb5kdc[4047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056684, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:51:24.449206 4062 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 22:51:24.449595 4062 webserver.cc:460] Webserver started at http://127.0.0.1:44873/ using document root <none> and password file <none>
I20260806 22:51:24.449772 4062 fs_manager.cc:359] Metadata directory not provided
I20260806 22:51:24.449812 4062 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:51:24.449896 4062 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:51:24.450871 4062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056405724609-6305-0/minicluster-data/master-0/data/instance:
uuid: "efa10887fa5b464eb8b75b9b6a28abad"
format_stamp: "Formatted at 2026-08-06 22:51:24 on dist-test-slave-zh2d"
server_key: "025de89b427a2fb6dbfa36479e545c45"
server_key_iv: "3a40784220046e39ad2648d33945dcd5"
server_key_version: "encryptionkey@0"
I20260806 22:51:24.451170 4062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056405724609-6305-0/minicluster-data/master-0/wal/instance:
uuid: "efa10887fa5b464eb8b75b9b6a28abad"
format_stamp: "Formatted at 2026-08-06 22:51:24 on dist-test-slave-zh2d"
server_key: "025de89b427a2fb6dbfa36479e545c45"
server_key_iv: "3a40784220046e39ad2648d33945dcd5"
server_key_version: "encryptionkey@0"
I20260806 22:51:24.452472 4062 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 22:51:24.453284 4082 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:51:24.453534 4062 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 22:51:24.453612 4062 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056405724609-6305-0/minicluster-data/master-0/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056405724609-6305-0/minicluster-data/master-0/wal
uuid: "efa10887fa5b464eb8b75b9b6a28abad"
format_stamp: "Formatted at 2026-08-06 22:51:24 on dist-test-slave-zh2d"
server_key: "025de89b427a2fb6dbfa36479e545c45"
server_key_iv: "3a40784220046e39ad2648d33945dcd5"
server_key_version: "encryptionkey@0"
I20260806 22:51:24.453703 4062 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056405724609-6305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056405724609-6305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056405724609-6305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:51:24.457022 4062 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:51:24.459313 4062 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:51:24.459445 4062 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:51:24.463496 4062 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44505
I20260806 22:51:24.463565 4154 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44505 every 8 connection(s)
I20260806 22:51:24.463878 4062 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
I20260806 22:51:24.464797 4155 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:51:24.467156 4155 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P efa10887fa5b464eb8b75b9b6a28abad: Bootstrap starting.
I20260806 22:51:24.467218 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 4062
I20260806 22:51:24.467370 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056405724609-6305-0/minicluster-data/master-0/wal/instance
I20260806 22:51:24.467581 6305 external_mini_cluster.cc:1428] Setting key 2877c2b16850059cf1d01c6db47e766f
I20260806 22:51:24.467892 4155 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P efa10887fa5b464eb8b75b9b6a28abad: Neither blocks nor log segments found. Creating new log.
I20260806 22:51:24.468253 4155 log.cc:824] T 00000000000000000000000000000000 P efa10887fa5b464eb8b75b9b6a28abad: Log is configured to *not* fsync() on all Append() calls
I20260806 22:51:24.469012 4155 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P efa10887fa5b464eb8b75b9b6a28abad: No bootstrap required, opened a new log
I20260806 22:51:24.470125 4158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:24.468054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.248:35689 (local address 127.0.0.1:44505)
0806 22:51:24.468273 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:24.468289 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:24.468340 (+ 51us) server_negotiation.cc:407] Connection header received
0806 22:51:24.468788 (+ 448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:24.468794 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:24.468821 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:24.469019 (+ 198us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:24.469708 (+ 689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:51:24.469712 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:51:24.469712 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:51:24.469716 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:51:24.469763 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:51:24.469772 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:51:24.469870 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:24.469910 (+ 40us) server_negotiation.cc:299] Negotiation successful
0806 22:51:24.469954 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":85,"threads_started":1}
I20260806 22:51:24.471082 6305 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:51:24.471481 4155 raft_consensus.cc:348] T 00000000000000000000000000000000 P efa10887fa5b464eb8b75b9b6a28abad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efa10887fa5b464eb8b75b9b6a28abad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44505 } }
I20260806 22:51:24.471634 4155 raft_consensus.cc:374] T 00000000000000000000000000000000 P efa10887fa5b464eb8b75b9b6a28abad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:51:24.471679 4155 raft_consensus.cc:729] T 00000000000000000000000000000000 P efa10887fa5b464eb8b75b9b6a28abad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa10887fa5b464eb8b75b9b6a28abad, State: Initialized, Role: FOLLOWER
I20260806 22:51:24.471805 4155 consensus_queue.cc:260] T 00000000000000000000000000000000 P efa10887fa5b464eb8b75b9b6a28abad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efa10887fa5b464eb8b75b9b6a28abad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44505 } }
I20260806 22:51:24.471894 4155 raft_consensus.cc:388] T 00000000000000000000000000000000 P efa10887fa5b464eb8b75b9b6a28abad [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:51:24.471938 4155 raft_consensus.cc:482] T 00000000000000000000000000000000 P efa10887fa5b464eb8b75b9b6a28abad [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:51:24.471987 4155 raft_consensus.cc:3037] T 00000000000000000000000000000000 P efa10887fa5b464eb8b75b9b6a28abad [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:51:24.472776 4155 raft_consensus.cc:504] T 00000000000000000000000000000000 P efa10887fa5b464eb8b75b9b6a28abad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efa10887fa5b464eb8b75b9b6a28abad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44505 } }
I20260806 22:51:24.472918 4155 leader_election.cc:302] T 00000000000000000000000000000000 P efa10887fa5b464eb8b75b9b6a28abad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: efa10887fa5b464eb8b75b9b6a28abad; no voters:
I20260806 22:51:24.473104 4155 leader_election.cc:288] T 00000000000000000000000000000000 P efa10887fa5b464eb8b75b9b6a28abad [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:51:24.473162 4167 raft_consensus.cc:2781] T 00000000000000000000000000000000 P efa10887fa5b464eb8b75b9b6a28abad [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:51:24.473341 4167 raft_consensus.cc:686] T 00000000000000000000000000000000 P efa10887fa5b464eb8b75b9b6a28abad [term 1 LEADER]: Becoming Leader. State: Replica: efa10887fa5b464eb8b75b9b6a28abad, State: Running, Role: LEADER
I20260806 22:51:24.473379 4155 sys_catalog.cc:564] T 00000000000000000000000000000000 P efa10887fa5b464eb8b75b9b6a28abad [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:51:24.473752 4158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:24.472634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.248:57485 (local address 127.0.0.1:44505)
0806 22:51:24.472692 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:24.472694 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:24.472703 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:51:24.472773 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:24.472774 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:24.472778 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:24.472832 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:24.473459 (+ 627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:51:24.473460 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:51:24.473460 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:51:24.473465 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:51:24.473478 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:51:24.473482 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:51:24.473538 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:24.473681 (+ 143us) server_negotiation.cc:299] Negotiation successful
0806 22:51:24.473693 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 22:51:24.474793 4167 consensus_queue.cc:237] T 00000000000000000000000000000000 P efa10887fa5b464eb8b75b9b6a28abad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efa10887fa5b464eb8b75b9b6a28abad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44505 } }
I20260806 22:51:24.475523 4168 sys_catalog.cc:455] T 00000000000000000000000000000000 P efa10887fa5b464eb8b75b9b6a28abad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "efa10887fa5b464eb8b75b9b6a28abad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efa10887fa5b464eb8b75b9b6a28abad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44505 } } }
I20260806 22:51:24.475391 4169 sys_catalog.cc:455] T 00000000000000000000000000000000 P efa10887fa5b464eb8b75b9b6a28abad [sys.catalog]: SysCatalogTable state changed. Reason: New leader efa10887fa5b464eb8b75b9b6a28abad. Latest consensus state: current_term: 1 leader_uuid: "efa10887fa5b464eb8b75b9b6a28abad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efa10887fa5b464eb8b75b9b6a28abad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44505 } } }
I20260806 22:51:24.475587 4168 sys_catalog.cc:458] T 00000000000000000000000000000000 P efa10887fa5b464eb8b75b9b6a28abad [sys.catalog]: This master's current role is: LEADER
I20260806 22:51:24.475600 4169 sys_catalog.cc:458] T 00000000000000000000000000000000 P efa10887fa5b464eb8b75b9b6a28abad [sys.catalog]: This master's current role is: LEADER
W20260806 22:51:24.476174 4188 catalog_manager.cc:1558] T 00000000000000000000000000000000 P efa10887fa5b464eb8b75b9b6a28abad: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:51:24.476240 4188 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:51:24.476301 4189 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:51:24.476482 4189 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:51:24.478082 4189 catalog_manager.cc:1347] Generated new cluster ID: 5c79ec82387f49a1a5d0f35d96b10d38
I20260806 22:51:24.478142 4189 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:51:24.488606 4189 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:51:24.488794 4189 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:51:24.495752 4189 catalog_manager.cc:5868] T 00000000000000000000000000000000 P efa10887fa5b464eb8b75b9b6a28abad: Generated new TSK 0
I20260806 22:51:24.495955 4189 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:51:24.499050 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 4062
2026-08-06T22:51:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1478 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056405724609-6305-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:51:24 dist-test-slave-zh2d krb5kdc[4197](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:51:24 dist-test-slave-zh2d krb5kdc[4197](info): set up 2 sockets
Aug 06 22:51:24 dist-test-slave-zh2d krb5kdc[4197](info): commencing operation
krb5kdc: starting...
W20260806 22:51:25.714031 6305 mini_kdc.cc:121] Time spent starting KDC: real 1.191s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056405724609-6305-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056405724609-6305-0/krb5kdc/test-user.keytab.
Aug 06 22:51:25 dist-test-slave-zh2d krb5kdc[4197](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056685, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:51:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:51:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.40.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.40.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.40.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab.
Entry for principal kudu/127.6.40.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab.
I20260806 22:51:25.844064 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056405724609-6305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056405724609-6305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056405724609-6305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.126:45611
--webserver_interface=127.6.40.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab
--principal=kudu/127.6.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:39057
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.40.126:45611
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:51:25.928746 4212 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:51:25.929064 4212 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:51:25.929109 4212 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:51:25.930281 4212 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:51:25.930353 4212 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:51:25.930379 4212 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:51:25.930400 4212 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:51:25.930428 4212 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:51:25.932431 4212 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:39057
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056405724609-6305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056405724609-6305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.40.126:45611
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab
--principal=kudu/127.6.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.126:45611
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4212
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056405724609-6305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:51:25.932825 4212 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:51:25.933133 4212 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:51:25.939721 4222 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:51:25.940201 4224 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:51:25.940586 4219 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:51:25.941107 4212 server_base.cc:1034] running on GCE node
I20260806 22:51:25.941417 4212 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:51:25.941792 4212 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:51:25.942983 4212 hybrid_clock.cc:648] HybridClock initialized: now 1786056685942954 us; error 54 us; skew 500 ppm
Aug 06 22:51:25 dist-test-slave-zh2d krb5kdc[4197](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056685, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:51:25.945586 4212 init.cc:377] Logged in from keytab as kudu/127.6.40.126@KRBTEST.COM (short username kudu)
I20260806 22:51:25.946063 4212 webserver.cc:460] Webserver started at http://127.6.40.126:39483/ using document root <none> and password file <none>
I20260806 22:51:25.946316 4212 fs_manager.cc:359] Metadata directory not provided
I20260806 22:51:25.946388 4212 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:51:25.946616 4212 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:51:25.948132 4212 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056405724609-6305-0/minicluster-data/master-0/data/instance:
uuid: "49c62828b3234f549e1854beb4005add"
format_stamp: "Formatted at 2026-08-06 22:51:25 on dist-test-slave-zh2d"
server_key: "7defea4149fddc6016fffa7629628a8c"
server_key_iv: "c714e15d538bf76196c5f740375f9b5c"
server_key_version: "encryptionkey@0"
I20260806 22:51:25.948506 4212 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056405724609-6305-0/minicluster-data/master-0/wal/instance:
uuid: "49c62828b3234f549e1854beb4005add"
format_stamp: "Formatted at 2026-08-06 22:51:25 on dist-test-slave-zh2d"
server_key: "7defea4149fddc6016fffa7629628a8c"
server_key_iv: "c714e15d538bf76196c5f740375f9b5c"
server_key_version: "encryptionkey@0"
I20260806 22:51:25.949994 4212 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260806 22:51:25.950757 4235 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:51:25.950949 4212 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:51:25.951061 4212 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056405724609-6305-0/minicluster-data/master-0/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056405724609-6305-0/minicluster-data/master-0/wal
uuid: "49c62828b3234f549e1854beb4005add"
format_stamp: "Formatted at 2026-08-06 22:51:25 on dist-test-slave-zh2d"
server_key: "7defea4149fddc6016fffa7629628a8c"
server_key_iv: "c714e15d538bf76196c5f740375f9b5c"
server_key_version: "encryptionkey@0"
I20260806 22:51:25.951140 4212 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056405724609-6305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056405724609-6305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056405724609-6305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:51:25.970881 4212 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:51:25.972884 4212 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:51:25.973085 4212 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:51:25.977311 4212 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.126:45611
I20260806 22:51:25.977349 4315 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.126:45611 every 8 connection(s)
I20260806 22:51:25.977734 4212 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
I20260806 22:51:25.978632 4316 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:51:25.980643 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 4212
I20260806 22:51:25.980773 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056405724609-6305-0/minicluster-data/master-0/wal/instance
I20260806 22:51:25.981004 6305 external_mini_cluster.cc:1428] Setting key 57c5c06b63d7f64a3cd5d05c0348a0a6
I20260806 22:51:25.981297 4316 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49c62828b3234f549e1854beb4005add: Bootstrap starting.
I20260806 22:51:25.982010 4316 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 49c62828b3234f549e1854beb4005add: Neither blocks nor log segments found. Creating new log.
I20260806 22:51:25.982357 4316 log.cc:824] T 00000000000000000000000000000000 P 49c62828b3234f549e1854beb4005add: Log is configured to *not* fsync() on all Append() calls
I20260806 22:51:25.983093 4316 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49c62828b3234f549e1854beb4005add: No bootstrap required, opened a new log
I20260806 22:51:25.984920 4316 raft_consensus.cc:348] T 00000000000000000000000000000000 P 49c62828b3234f549e1854beb4005add [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49c62828b3234f549e1854beb4005add" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 45611 } }
I20260806 22:51:25.985087 4316 raft_consensus.cc:374] T 00000000000000000000000000000000 P 49c62828b3234f549e1854beb4005add [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:51:25.985136 4316 raft_consensus.cc:729] T 00000000000000000000000000000000 P 49c62828b3234f549e1854beb4005add [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49c62828b3234f549e1854beb4005add, State: Initialized, Role: FOLLOWER
I20260806 22:51:25.985312 4316 consensus_queue.cc:260] T 00000000000000000000000000000000 P 49c62828b3234f549e1854beb4005add [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49c62828b3234f549e1854beb4005add" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 45611 } }
I20260806 22:51:25.985410 4316 raft_consensus.cc:388] T 00000000000000000000000000000000 P 49c62828b3234f549e1854beb4005add [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:51:25.985451 4316 raft_consensus.cc:482] T 00000000000000000000000000000000 P 49c62828b3234f549e1854beb4005add [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:51:25.985496 4316 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 49c62828b3234f549e1854beb4005add [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:51:25.986203 4316 raft_consensus.cc:504] T 00000000000000000000000000000000 P 49c62828b3234f549e1854beb4005add [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49c62828b3234f549e1854beb4005add" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 45611 } }
I20260806 22:51:25.986330 4316 leader_election.cc:302] T 00000000000000000000000000000000 P 49c62828b3234f549e1854beb4005add [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 49c62828b3234f549e1854beb4005add; no voters:
I20260806 22:51:25.986522 4316 leader_election.cc:288] T 00000000000000000000000000000000 P 49c62828b3234f549e1854beb4005add [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:51:25.986553 4323 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 49c62828b3234f549e1854beb4005add [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:51:25.986676 4323 raft_consensus.cc:686] T 00000000000000000000000000000000 P 49c62828b3234f549e1854beb4005add [term 1 LEADER]: Becoming Leader. State: Replica: 49c62828b3234f549e1854beb4005add, State: Running, Role: LEADER
I20260806 22:51:25.986761 4323 consensus_queue.cc:237] T 00000000000000000000000000000000 P 49c62828b3234f549e1854beb4005add [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49c62828b3234f549e1854beb4005add" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 45611 } }
I20260806 22:51:25.986792 4316 sys_catalog.cc:564] T 00000000000000000000000000000000 P 49c62828b3234f549e1854beb4005add [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:51:25.987299 4328 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49c62828b3234f549e1854beb4005add [sys.catalog]: SysCatalogTable state changed. Reason: New leader 49c62828b3234f549e1854beb4005add. Latest consensus state: current_term: 1 leader_uuid: "49c62828b3234f549e1854beb4005add" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49c62828b3234f549e1854beb4005add" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 45611 } } }
I20260806 22:51:25.987452 4328 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49c62828b3234f549e1854beb4005add [sys.catalog]: This master's current role is: LEADER
Aug 06 22:51:25 dist-test-slave-zh2d krb5kdc[4197](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056685, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
I20260806 22:51:25.987929 4326 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49c62828b3234f549e1854beb4005add [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "49c62828b3234f549e1854beb4005add" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49c62828b3234f549e1854beb4005add" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 45611 } } }
I20260806 22:51:25.987979 4337 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:51:25.988034 4326 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49c62828b3234f549e1854beb4005add [sys.catalog]: This master's current role is: LEADER
I20260806 22:51:25.988453 4337 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:51:25.990235 4337 catalog_manager.cc:1347] Generated new cluster ID: 40e0293662264913846f5d61e42acf3e
I20260806 22:51:25.990288 4337 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:51:25.992938 4319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:25.981539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33093 (local address 127.6.40.126:45611)
0806 22:51:25.981757 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:25.981768 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:25.981805 (+ 37us) server_negotiation.cc:407] Connection header received
0806 22:51:25.982299 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:25.982307 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:25.982354 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:25.982510 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:25.983191 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:25.984085 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:25.984869 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:25.985363 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:25.985494 (+ 131us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:25.988393 (+ 2899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:51:25.988396 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:51:25.988397 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:51:25.988407 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:51:25.990747 (+ 2340us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:25.991234 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:25.991236 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:25.991237 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:25.991341 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:25.991617 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:25.991617 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:25.991618 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:25.991932 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:51:25.992098 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:25.992574 (+ 476us) server_negotiation.cc:299] Negotiation successful
0806 22:51:25.992689 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":106,"threads_started":1}
I20260806 22:51:25.993997 6305 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:51:26.000802 4337 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:51:26.001654 4337 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:51:26.003980 4319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:25.995592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54475 (local address 127.6.40.126:45611)
0806 22:51:25.995635 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:25.995637 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:25.995649 (+ 12us) server_negotiation.cc:407] Connection header received
0806 22:51:25.995699 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:25.995699 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:25.995711 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:25.995793 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:25.996364 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:25.997149 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:25.998102 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:25.998446 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:25.998523 (+ 77us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:25.999879 (+ 1356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:51:25.999881 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:51:25.999881 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:51:25.999888 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:51:26.002585 (+ 2697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:26.002991 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:26.002992 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:26.002992 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:26.003040 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:26.003255 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:26.003256 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:26.003256 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:26.003359 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:51:26.003788 (+ 429us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:26.003837 (+ 49us) server_negotiation.cc:299] Negotiation successful
0806 22:51:26.003895 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":9,"server-negotiator.queue_time_us":22}
I20260806 22:51:26.005962 4337 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 49c62828b3234f549e1854beb4005add: Generated new TSK 0
I20260806 22:51:26.006136 4337 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:51:26.009320 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 4212
W20260806 22:51:26.022768 4195 connection.cc:535] client connection to 127.6.40.126:45611 recv error: Network error: failed to read from TLS socket (remote: 127.6.40.126:45611): Connection reset by peer (error 104)
2026-08-06T22:51:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1513 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056405724609-6305-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:51:26 dist-test-slave-zh2d krb5kdc[4359](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:51:26 dist-test-slave-zh2d krb5kdc[4359](info): set up 2 sockets
Aug 06 22:51:26 dist-test-slave-zh2d krb5kdc[4359](info): commencing operation
krb5kdc: starting...
W20260806 22:51:27.229313 6305 mini_kdc.cc:121] Time spent starting KDC: real 1.193s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056405724609-6305-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056405724609-6305-0/krb5kdc/test-user.keytab.
Aug 06 22:51:27 dist-test-slave-zh2d krb5kdc[4359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056687, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:51:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:51:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.40.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.40.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.40.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab.
Entry for principal kudu/127.6.40.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab.
I20260806 22:51:27.355870 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056405724609-6305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056405724609-6305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056405724609-6305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.126:34087
--webserver_interface=127.6.40.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab
--principal=kudu/127.6.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:44395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.40.126:34087
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:51:27.438941 4378 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:51:27.439183 4378 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:51:27.439224 4378 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:51:27.440341 4378 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:51:27.440413 4378 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:51:27.440438 4378 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:51:27.440467 4378 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:51:27.440508 4378 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:51:27.442045 4378 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:44395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056405724609-6305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056405724609-6305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.40.126:34087
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab
--principal=kudu/127.6.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.126:34087
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4378
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056405724609-6305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:51:27.442322 4378 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:51:27.442562 4378 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:51:27.448264 4386 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:51:27.448235 4391 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:51:27.448253 4385 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:51:27.448588 4378 server_base.cc:1034] running on GCE node
I20260806 22:51:27.448930 4378 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:51:27.449240 4378 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:51:27.450416 4378 hybrid_clock.cc:648] HybridClock initialized: now 1786056687450407 us; error 40 us; skew 500 ppm
Aug 06 22:51:27 dist-test-slave-zh2d krb5kdc[4359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056687, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:51:27.453024 4378 init.cc:377] Logged in from keytab as kudu/127.6.40.126@KRBTEST.COM (short username kudu)
I20260806 22:51:27.453481 4378 webserver.cc:460] Webserver started at http://127.6.40.126:39131/ using document root <none> and password file <none>
I20260806 22:51:27.453740 4378 fs_manager.cc:359] Metadata directory not provided
I20260806 22:51:27.453814 4378 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:51:27.453970 4378 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:51:27.455039 4378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056405724609-6305-0/minicluster-data/master-0/data/instance:
uuid: "1d60d3a0be04439693b978b95aefdfc6"
format_stamp: "Formatted at 2026-08-06 22:51:27 on dist-test-slave-zh2d"
server_key: "cca7883d8475c671baa6e5df4b70cb77"
server_key_iv: "7c01df3dda798382a5d30fe660cfc135"
server_key_version: "encryptionkey@0"
I20260806 22:51:27.455408 4378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056405724609-6305-0/minicluster-data/master-0/wal/instance:
uuid: "1d60d3a0be04439693b978b95aefdfc6"
format_stamp: "Formatted at 2026-08-06 22:51:27 on dist-test-slave-zh2d"
server_key: "cca7883d8475c671baa6e5df4b70cb77"
server_key_iv: "7c01df3dda798382a5d30fe660cfc135"
server_key_version: "encryptionkey@0"
I20260806 22:51:27.456765 4378 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 22:51:27.457551 4401 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:51:27.457764 4378 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:51:27.457883 4378 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056405724609-6305-0/minicluster-data/master-0/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056405724609-6305-0/minicluster-data/master-0/wal
uuid: "1d60d3a0be04439693b978b95aefdfc6"
format_stamp: "Formatted at 2026-08-06 22:51:27 on dist-test-slave-zh2d"
server_key: "cca7883d8475c671baa6e5df4b70cb77"
server_key_iv: "7c01df3dda798382a5d30fe660cfc135"
server_key_version: "encryptionkey@0"
I20260806 22:51:27.457974 4378 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056405724609-6305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056405724609-6305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056405724609-6305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:51:27.472429 4378 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:51:27.474468 4378 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:51:27.474642 4378 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:51:27.478790 4378 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.126:34087
I20260806 22:51:27.478843 4479 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.126:34087 every 8 connection(s)
I20260806 22:51:27.479228 4378 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
I20260806 22:51:27.480129 4480 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:51:27.481393 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 4378
I20260806 22:51:27.481532 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056405724609-6305-0/minicluster-data/master-0/wal/instance
I20260806 22:51:27.481737 6305 external_mini_cluster.cc:1428] Setting key e68da217ae5fec5b908ccff5615ae15d
I20260806 22:51:27.483026 4480 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d60d3a0be04439693b978b95aefdfc6: Bootstrap starting.
I20260806 22:51:27.483783 4480 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1d60d3a0be04439693b978b95aefdfc6: Neither blocks nor log segments found. Creating new log.
I20260806 22:51:27.484133 4480 log.cc:824] T 00000000000000000000000000000000 P 1d60d3a0be04439693b978b95aefdfc6: Log is configured to *not* fsync() on all Append() calls
I20260806 22:51:27.485018 4480 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d60d3a0be04439693b978b95aefdfc6: No bootstrap required, opened a new log
I20260806 22:51:27.487360 4480 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1d60d3a0be04439693b978b95aefdfc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d60d3a0be04439693b978b95aefdfc6" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 34087 } }
I20260806 22:51:27.487542 4480 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1d60d3a0be04439693b978b95aefdfc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:51:27.487586 4480 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1d60d3a0be04439693b978b95aefdfc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d60d3a0be04439693b978b95aefdfc6, State: Initialized, Role: FOLLOWER
I20260806 22:51:27.487726 4480 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1d60d3a0be04439693b978b95aefdfc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d60d3a0be04439693b978b95aefdfc6" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 34087 } }
I20260806 22:51:27.487819 4480 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1d60d3a0be04439693b978b95aefdfc6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:51:27.487852 4480 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1d60d3a0be04439693b978b95aefdfc6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:51:27.487897 4480 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1d60d3a0be04439693b978b95aefdfc6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:51:27.488221 4483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:27.482303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46745 (local address 127.6.40.126:34087)
0806 22:51:27.482523 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:27.482534 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:27.482573 (+ 39us) server_negotiation.cc:407] Connection header received
0806 22:51:27.483139 (+ 566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:27.483148 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:27.483210 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:27.483366 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:27.483619 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:27.484965 (+ 1346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:27.485905 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:27.486536 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:27.486668 (+ 132us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:27.487398 (+ 730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:51:27.487403 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:51:27.487403 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:51:27.487625 (+ 222us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:51:27.487667 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:51:27.487682 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:51:27.487770 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:27.488001 (+ 231us) server_negotiation.cc:299] Negotiation successful
0806 22:51:27.488040 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":90,"threads_started":1}
I20260806 22:51:27.488575 4480 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1d60d3a0be04439693b978b95aefdfc6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d60d3a0be04439693b978b95aefdfc6" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 34087 } }
I20260806 22:51:27.488684 4480 leader_election.cc:302] T 00000000000000000000000000000000 P 1d60d3a0be04439693b978b95aefdfc6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1d60d3a0be04439693b978b95aefdfc6; no voters:
I20260806 22:51:27.488858 4480 leader_election.cc:288] T 00000000000000000000000000000000 P 1d60d3a0be04439693b978b95aefdfc6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:51:27.488881 4485 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1d60d3a0be04439693b978b95aefdfc6 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:51:27.489092 4485 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1d60d3a0be04439693b978b95aefdfc6 [term 1 LEADER]: Becoming Leader. State: Replica: 1d60d3a0be04439693b978b95aefdfc6, State: Running, Role: LEADER
I20260806 22:51:27.489223 4480 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1d60d3a0be04439693b978b95aefdfc6 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:51:27.489296 4485 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1d60d3a0be04439693b978b95aefdfc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d60d3a0be04439693b978b95aefdfc6" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 34087 } }
I20260806 22:51:27.489387 6305 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:51:27.489619 4485 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d60d3a0be04439693b978b95aefdfc6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1d60d3a0be04439693b978b95aefdfc6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d60d3a0be04439693b978b95aefdfc6" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 34087 } } }
I20260806 22:51:27.489776 4485 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d60d3a0be04439693b978b95aefdfc6 [sys.catalog]: This master's current role is: LEADER
I20260806 22:51:27.489984 4485 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d60d3a0be04439693b978b95aefdfc6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1d60d3a0be04439693b978b95aefdfc6. Latest consensus state: current_term: 1 leader_uuid: "1d60d3a0be04439693b978b95aefdfc6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d60d3a0be04439693b978b95aefdfc6" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 34087 } } }
I20260806 22:51:27.490094 4485 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d60d3a0be04439693b978b95aefdfc6 [sys.catalog]: This master's current role is: LEADER
I20260806 22:51:27.490466 4497 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:51:27.490994 4497 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:51:27.493507 4497 catalog_manager.cc:1347] Generated new cluster ID: 9bf201dde067444badfb576200cfdd0c
I20260806 22:51:27.493608 4497 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:51:27.495008 4483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:27.491259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50241 (local address 127.6.40.126:34087)
0806 22:51:27.491296 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:27.491298 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:27.491398 (+ 100us) server_negotiation.cc:407] Connection header received
0806 22:51:27.491521 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:27.491522 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:27.491532 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:27.491589 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:27.491764 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:27.492460 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:27.493567 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:27.493880 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:27.493955 (+ 75us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:27.494741 (+ 786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:51:27.494743 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:51:27.494743 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:51:27.494748 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:51:27.494765 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:51:27.494769 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:51:27.494837 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:27.494922 (+ 85us) server_negotiation.cc:299] Negotiation successful
0806 22:51:27.494933 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 22:51:27.509325 4497 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:51:27.509909 4497 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:51:27.512532 4497 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1d60d3a0be04439693b978b95aefdfc6: Generated new TSK 0
I20260806 22:51:27.512729 4497 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:51:27.516664 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 4378
W20260806 22:51:27.528070 4355 connection.cc:535] client connection to 127.6.40.126:34087 recv error: Network error: failed to read from TLS socket (remote: 127.6.40.126:34087): Connection reset by peer (error 104)
2026-08-06T22:51:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1505 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056405724609-6305-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:51:27 dist-test-slave-zh2d krb5kdc[4533](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:51:27 dist-test-slave-zh2d krb5kdc[4533](info): set up 2 sockets
Aug 06 22:51:27 dist-test-slave-zh2d krb5kdc[4533](info): commencing operation
krb5kdc: starting...
W20260806 22:51:28.747339 6305 mini_kdc.cc:121] Time spent starting KDC: real 1.206s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056405724609-6305-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056405724609-6305-0/krb5kdc/test-user.keytab.
Aug 06 22:51:28 dist-test-slave-zh2d krb5kdc[4533](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056688, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:51:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:51:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.40.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.40.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.40.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab.
Entry for principal kudu/127.6.40.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab.
I20260806 22:51:28.874027 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056405724609-6305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056405724609-6305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056405724609-6305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.126:38175
--webserver_interface=127.6.40.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab
--principal=kudu/127.6.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:42735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.40.126:38175
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:51:28.957990 4551 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:51:28.958228 4551 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:51:28.958266 4551 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:51:28.959388 4551 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:51:28.959460 4551 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:51:28.959494 4551 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:51:28.959522 4551 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:51:28.959549 4551 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:51:28.961077 4551 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:42735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056405724609-6305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056405724609-6305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.40.126:38175
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab
--principal=kudu/127.6.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.126:38175
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4551
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056405724609-6305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:51:28.961344 4551 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:51:28.961586 4551 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:51:28.966495 4561 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:51:28.966495 4558 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:51:28.966536 4557 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:51:28.966747 4551 server_base.cc:1034] running on GCE node
I20260806 22:51:28.966995 4551 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:51:28.967319 4551 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:51:28.968480 4551 hybrid_clock.cc:648] HybridClock initialized: now 1786056688968474 us; error 39 us; skew 500 ppm
Aug 06 22:51:28 dist-test-slave-zh2d krb5kdc[4533](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056688, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:51:28.971103 4551 init.cc:377] Logged in from keytab as kudu/127.6.40.126@KRBTEST.COM (short username kudu)
I20260806 22:51:28.971490 4551 webserver.cc:460] Webserver started at http://127.6.40.126:42485/ using document root <none> and password file <none>
I20260806 22:51:28.971688 4551 fs_manager.cc:359] Metadata directory not provided
I20260806 22:51:28.971728 4551 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:51:28.971815 4551 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:51:28.972792 4551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056405724609-6305-0/minicluster-data/master-0/data/instance:
uuid: "3964544493ad48c98967b7ca8aa0e678"
format_stamp: "Formatted at 2026-08-06 22:51:28 on dist-test-slave-zh2d"
server_key: "3f0cbac148a88802d4b7c7e47908cd43"
server_key_iv: "1d093db5c202171720dd440014ae2b37"
server_key_version: "encryptionkey@0"
I20260806 22:51:28.973194 4551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056405724609-6305-0/minicluster-data/master-0/wal/instance:
uuid: "3964544493ad48c98967b7ca8aa0e678"
format_stamp: "Formatted at 2026-08-06 22:51:28 on dist-test-slave-zh2d"
server_key: "3f0cbac148a88802d4b7c7e47908cd43"
server_key_iv: "1d093db5c202171720dd440014ae2b37"
server_key_version: "encryptionkey@0"
I20260806 22:51:28.974481 4551 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:51:28.975154 4572 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:51:28.975322 4551 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:51:28.975452 4551 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056405724609-6305-0/minicluster-data/master-0/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056405724609-6305-0/minicluster-data/master-0/wal
uuid: "3964544493ad48c98967b7ca8aa0e678"
format_stamp: "Formatted at 2026-08-06 22:51:28 on dist-test-slave-zh2d"
server_key: "3f0cbac148a88802d4b7c7e47908cd43"
server_key_iv: "1d093db5c202171720dd440014ae2b37"
server_key_version: "encryptionkey@0"
I20260806 22:51:28.975520 4551 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056405724609-6305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056405724609-6305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056405724609-6305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:51:28.978870 4551 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:51:28.980710 4551 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:51:28.980914 4551 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:51:28.984978 4551 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.126:38175
I20260806 22:51:28.985059 4645 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.126:38175 every 8 connection(s)
I20260806 22:51:28.985345 4551 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
I20260806 22:51:28.986337 4646 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:51:28.988577 4646 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3964544493ad48c98967b7ca8aa0e678: Bootstrap starting.
I20260806 22:51:28.989064 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 4551
I20260806 22:51:28.989198 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056405724609-6305-0/minicluster-data/master-0/wal/instance
I20260806 22:51:28.989298 4646 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3964544493ad48c98967b7ca8aa0e678: Neither blocks nor log segments found. Creating new log.
I20260806 22:51:28.989426 6305 external_mini_cluster.cc:1428] Setting key 152690eb6282a228fe9dedce5322e769
I20260806 22:51:28.989655 4646 log.cc:824] T 00000000000000000000000000000000 P 3964544493ad48c98967b7ca8aa0e678: Log is configured to *not* fsync() on all Append() calls
I20260806 22:51:28.990424 4646 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3964544493ad48c98967b7ca8aa0e678: No bootstrap required, opened a new log
I20260806 22:51:28.991906 4651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:28.989910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42129 (local address 127.6.40.126:38175)
0806 22:51:28.990070 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:28.990080 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:28.990117 (+ 37us) server_negotiation.cc:407] Connection header received
0806 22:51:28.990653 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:28.990660 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:28.990688 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:28.990805 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:28.991454 (+ 649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:51:28.991457 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:51:28.991457 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:51:28.991461 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:51:28.991498 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:51:28.991508 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:51:28.991578 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:28.991650 (+ 72us) server_negotiation.cc:299] Negotiation successful
0806 22:51:28.991682 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":73,"threads_started":1}
I20260806 22:51:28.992980 6305 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:51:28.993140 4646 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3964544493ad48c98967b7ca8aa0e678 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3964544493ad48c98967b7ca8aa0e678" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 38175 } }
I20260806 22:51:28.993297 4646 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3964544493ad48c98967b7ca8aa0e678 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:51:28.993327 4646 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3964544493ad48c98967b7ca8aa0e678 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3964544493ad48c98967b7ca8aa0e678, State: Initialized, Role: FOLLOWER
I20260806 22:51:28.993430 4646 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3964544493ad48c98967b7ca8aa0e678 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3964544493ad48c98967b7ca8aa0e678" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 38175 } }
I20260806 22:51:28.993535 4646 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3964544493ad48c98967b7ca8aa0e678 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:51:28.993569 4646 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3964544493ad48c98967b7ca8aa0e678 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:51:28.993620 4646 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3964544493ad48c98967b7ca8aa0e678 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:51:28.994402 4646 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3964544493ad48c98967b7ca8aa0e678 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3964544493ad48c98967b7ca8aa0e678" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 38175 } }
I20260806 22:51:28.994526 4646 leader_election.cc:302] T 00000000000000000000000000000000 P 3964544493ad48c98967b7ca8aa0e678 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3964544493ad48c98967b7ca8aa0e678; no voters:
I20260806 22:51:28.994690 4646 leader_election.cc:288] T 00000000000000000000000000000000 P 3964544493ad48c98967b7ca8aa0e678 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:51:28.994742 4656 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3964544493ad48c98967b7ca8aa0e678 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:51:28.994891 4656 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3964544493ad48c98967b7ca8aa0e678 [term 1 LEADER]: Becoming Leader. State: Replica: 3964544493ad48c98967b7ca8aa0e678, State: Running, Role: LEADER
I20260806 22:51:28.994992 4646 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3964544493ad48c98967b7ca8aa0e678 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:51:28.995030 4656 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3964544493ad48c98967b7ca8aa0e678 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3964544493ad48c98967b7ca8aa0e678" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 38175 } }
I20260806 22:51:28.995363 4656 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3964544493ad48c98967b7ca8aa0e678 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3964544493ad48c98967b7ca8aa0e678. Latest consensus state: current_term: 1 leader_uuid: "3964544493ad48c98967b7ca8aa0e678" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3964544493ad48c98967b7ca8aa0e678" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 38175 } } }
I20260806 22:51:28.995378 4659 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3964544493ad48c98967b7ca8aa0e678 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3964544493ad48c98967b7ca8aa0e678" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3964544493ad48c98967b7ca8aa0e678" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 38175 } } }
I20260806 22:51:28.995529 4656 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3964544493ad48c98967b7ca8aa0e678 [sys.catalog]: This master's current role is: LEADER
I20260806 22:51:28.995936 4659 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3964544493ad48c98967b7ca8aa0e678 [sys.catalog]: This master's current role is: LEADER
I20260806 22:51:28.996021 4651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:28.994682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39485 (local address 127.6.40.126:38175)
0806 22:51:28.994717 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:28.994718 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:28.994894 (+ 176us) server_negotiation.cc:407] Connection header received
0806 22:51:28.994982 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:28.994983 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:28.994988 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:28.995029 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:28.995752 (+ 723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:51:28.995753 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:51:28.995753 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:51:28.995758 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:51:28.995773 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:51:28.995778 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:51:28.995837 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:28.995945 (+ 108us) server_negotiation.cc:299] Negotiation successful
0806 22:51:28.995960 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 22:51:28.996707 4670 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:51:28.997355 4670 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:51:28.998749 4670 catalog_manager.cc:1347] Generated new cluster ID: a9b2b805d1534595ac5b44d5f69b653b
I20260806 22:51:28.998801 4670 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:51:29.008168 4670 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:51:29.008349 4670 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:51:29.011987 4670 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3964544493ad48c98967b7ca8aa0e678: Generated new TSK 0
I20260806 22:51:29.012173 4670 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:51:29.015617 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 4551
2026-08-06T22:51:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1499 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056405724609-6305-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:51:29 dist-test-slave-zh2d krb5kdc[4687](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:51:29 dist-test-slave-zh2d krb5kdc[4687](info): set up 2 sockets
Aug 06 22:51:29 dist-test-slave-zh2d krb5kdc[4687](info): commencing operation
krb5kdc: starting...
W20260806 22:51:30.213779 6305 mini_kdc.cc:121] Time spent starting KDC: real 1.172s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056405724609-6305-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056405724609-6305-0/krb5kdc/test-user.keytab.
Aug 06 22:51:30 dist-test-slave-zh2d krb5kdc[4687](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056690, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:51:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:51:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.40.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.40.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.40.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab.
Entry for principal kudu/127.6.40.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab.
I20260806 22:51:30.340031 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056405724609-6305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056405724609-6305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056405724609-6305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.126:46869
--webserver_interface=127.6.40.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab
--principal=kudu/127.6.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:34429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.40.126:46869
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:51:30.424355 4706 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:51:30.424619 4706 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:51:30.424661 4706 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:51:30.425851 4706 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:51:30.425927 4706 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:51:30.425951 4706 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:51:30.425972 4706 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:51:30.426010 4706 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:51:30.427531 4706 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:34429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056405724609-6305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056405724609-6305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.40.126:46869
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab
--principal=kudu/127.6.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.126:46869
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4706
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056405724609-6305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:51:30.427836 4706 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:51:30.428095 4706 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:51:30.433243 4712 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:51:30.433274 4711 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:51:30.433422 4706 server_base.cc:1034] running on GCE node
W20260806 22:51:30.433297 4715 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:51:30.433790 4706 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:51:30.434096 4706 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:51:30.435261 4706 hybrid_clock.cc:648] HybridClock initialized: now 1786056690435246 us; error 30 us; skew 500 ppm
Aug 06 22:51:30 dist-test-slave-zh2d krb5kdc[4687](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056690, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:51:30.437920 4706 init.cc:377] Logged in from keytab as kudu/127.6.40.126@KRBTEST.COM (short username kudu)
I20260806 22:51:30.438320 4706 webserver.cc:460] Webserver started at http://127.6.40.126:39953/ using document root <none> and password file <none>
I20260806 22:51:30.438506 4706 fs_manager.cc:359] Metadata directory not provided
I20260806 22:51:30.438547 4706 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:51:30.438634 4706 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:51:30.439652 4706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056405724609-6305-0/minicluster-data/master-0/data/instance:
uuid: "ada75c6549144c699dbed1844a24d91d"
format_stamp: "Formatted at 2026-08-06 22:51:30 on dist-test-slave-zh2d"
server_key: "6461e22b6bbf95af74544a19bae6ada3"
server_key_iv: "363a53c2fd21399060e07dec8b98f9e9"
server_key_version: "encryptionkey@0"
I20260806 22:51:30.439968 4706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056405724609-6305-0/minicluster-data/master-0/wal/instance:
uuid: "ada75c6549144c699dbed1844a24d91d"
format_stamp: "Formatted at 2026-08-06 22:51:30 on dist-test-slave-zh2d"
server_key: "6461e22b6bbf95af74544a19bae6ada3"
server_key_iv: "363a53c2fd21399060e07dec8b98f9e9"
server_key_version: "encryptionkey@0"
I20260806 22:51:30.441426 4706 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:51:30.442076 4727 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:51:30.442252 4706 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:51:30.442359 4706 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056405724609-6305-0/minicluster-data/master-0/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056405724609-6305-0/minicluster-data/master-0/wal
uuid: "ada75c6549144c699dbed1844a24d91d"
format_stamp: "Formatted at 2026-08-06 22:51:30 on dist-test-slave-zh2d"
server_key: "6461e22b6bbf95af74544a19bae6ada3"
server_key_iv: "363a53c2fd21399060e07dec8b98f9e9"
server_key_version: "encryptionkey@0"
I20260806 22:51:30.442436 4706 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056405724609-6305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056405724609-6305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056405724609-6305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:51:30.451665 4706 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:51:30.454172 4706 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:51:30.454344 4706 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:51:30.458652 4706 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.126:46869
I20260806 22:51:30.458786 4799 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.126:46869 every 8 connection(s)
I20260806 22:51:30.459115 4706 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
I20260806 22:51:30.460040 4801 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:51:30.462525 4801 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ada75c6549144c699dbed1844a24d91d: Bootstrap starting.
I20260806 22:51:30.463143 4801 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ada75c6549144c699dbed1844a24d91d: Neither blocks nor log segments found. Creating new log.
I20260806 22:51:30.463443 4801 log.cc:824] T 00000000000000000000000000000000 P ada75c6549144c699dbed1844a24d91d: Log is configured to *not* fsync() on all Append() calls
I20260806 22:51:30.464174 4801 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ada75c6549144c699dbed1844a24d91d: No bootstrap required, opened a new log
I20260806 22:51:30.465978 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 4706
I20260806 22:51:30.465921 4801 raft_consensus.cc:348] T 00000000000000000000000000000000 P ada75c6549144c699dbed1844a24d91d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ada75c6549144c699dbed1844a24d91d" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 46869 } }
I20260806 22:51:30.466084 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056405724609-6305-0/minicluster-data/master-0/wal/instance
I20260806 22:51:30.466082 4801 raft_consensus.cc:374] T 00000000000000000000000000000000 P ada75c6549144c699dbed1844a24d91d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:51:30.466156 4801 raft_consensus.cc:729] T 00000000000000000000000000000000 P ada75c6549144c699dbed1844a24d91d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ada75c6549144c699dbed1844a24d91d, State: Initialized, Role: FOLLOWER
I20260806 22:51:30.466326 4801 consensus_queue.cc:260] T 00000000000000000000000000000000 P ada75c6549144c699dbed1844a24d91d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ada75c6549144c699dbed1844a24d91d" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 46869 } }
I20260806 22:51:30.466364 6305 external_mini_cluster.cc:1428] Setting key 4e4bc8014195bf855e7e603390cc8789
I20260806 22:51:30.466413 4801 raft_consensus.cc:388] T 00000000000000000000000000000000 P ada75c6549144c699dbed1844a24d91d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:51:30.466459 4801 raft_consensus.cc:482] T 00000000000000000000000000000000 P ada75c6549144c699dbed1844a24d91d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:51:30.466507 4801 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ada75c6549144c699dbed1844a24d91d [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:51:30.467214 4801 raft_consensus.cc:504] T 00000000000000000000000000000000 P ada75c6549144c699dbed1844a24d91d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ada75c6549144c699dbed1844a24d91d" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 46869 } }
I20260806 22:51:30.467343 4801 leader_election.cc:302] T 00000000000000000000000000000000 P ada75c6549144c699dbed1844a24d91d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ada75c6549144c699dbed1844a24d91d; no voters:
I20260806 22:51:30.467530 4801 leader_election.cc:288] T 00000000000000000000000000000000 P ada75c6549144c699dbed1844a24d91d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:51:30.467553 4809 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ada75c6549144c699dbed1844a24d91d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:51:30.467670 4809 raft_consensus.cc:686] T 00000000000000000000000000000000 P ada75c6549144c699dbed1844a24d91d [term 1 LEADER]: Becoming Leader. State: Replica: ada75c6549144c699dbed1844a24d91d, State: Running, Role: LEADER
I20260806 22:51:30.467840 4801 sys_catalog.cc:564] T 00000000000000000000000000000000 P ada75c6549144c699dbed1844a24d91d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:51:30.467823 4809 consensus_queue.cc:237] T 00000000000000000000000000000000 P ada75c6549144c699dbed1844a24d91d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ada75c6549144c699dbed1844a24d91d" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 46869 } }
I20260806 22:51:30.468302 4811 sys_catalog.cc:455] T 00000000000000000000000000000000 P ada75c6549144c699dbed1844a24d91d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ada75c6549144c699dbed1844a24d91d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ada75c6549144c699dbed1844a24d91d" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 46869 } } }
I20260806 22:51:30.468410 4811 sys_catalog.cc:458] T 00000000000000000000000000000000 P ada75c6549144c699dbed1844a24d91d [sys.catalog]: This master's current role is: LEADER
I20260806 22:51:30.468636 4810 sys_catalog.cc:455] T 00000000000000000000000000000000 P ada75c6549144c699dbed1844a24d91d [sys.catalog]: SysCatalogTable state changed. Reason: New leader ada75c6549144c699dbed1844a24d91d. Latest consensus state: current_term: 1 leader_uuid: "ada75c6549144c699dbed1844a24d91d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ada75c6549144c699dbed1844a24d91d" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 46869 } } }
I20260806 22:51:30.469010 4810 sys_catalog.cc:458] T 00000000000000000000000000000000 P ada75c6549144c699dbed1844a24d91d [sys.catalog]: This master's current role is: LEADER
I20260806 22:51:30.469210 4817 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:51:30.469738 4817 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:51:30.471232 4817 catalog_manager.cc:1347] Generated new cluster ID: ea75768a86c6422391dd178d5d2e07a9
I20260806 22:51:30.471284 4817 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 22:51:30 dist-test-slave-zh2d krb5kdc[4687](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056690, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
I20260806 22:51:30.479099 4806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:30.466935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40435 (local address 127.6.40.126:46869)
0806 22:51:30.467525 (+ 590us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:30.467534 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:30.467560 (+ 26us) server_negotiation.cc:407] Connection header received
0806 22:51:30.467706 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:30.467714 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:30.467769 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:30.468769 (+ 1000us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:30.468947 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:30.470510 (+ 1563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:30.471618 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:30.472253 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:30.472375 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:30.475345 (+ 2970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:51:30.475350 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:51:30.475350 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:51:30.475363 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:51:30.477399 (+ 2036us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:30.477825 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:30.477834 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:30.477835 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:30.477947 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:30.478174 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:30.478174 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:30.478175 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:30.478464 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:51:30.478606 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:30.478743 (+ 137us) server_negotiation.cc:299] Negotiation successful
0806 22:51:30.478881 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":531,"thread_start_us":478,"threads_started":1}
I20260806 22:51:30.480386 6305 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:51:30.493034 4817 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:51:30.493777 4817 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:51:30.500237 4806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:30.485166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49399 (local address 127.6.40.126:46869)
0806 22:51:30.485206 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:30.485208 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:30.492610 (+ 7402us) server_negotiation.cc:407] Connection header received
0806 22:51:30.492648 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:30.492649 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:30.492664 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:30.492740 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:30.493541 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:30.494328 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:30.495350 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:30.495742 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:30.495819 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:30.497278 (+ 1459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:51:30.497280 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:51:30.497280 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:51:30.497287 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:51:30.499012 (+ 1725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:30.499419 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:30.499420 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:30.499420 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:30.499486 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:30.499691 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:30.499692 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:30.499693 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:30.499827 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:51:30.499936 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:30.500030 (+ 94us) server_negotiation.cc:299] Negotiation successful
0806 22:51:30.500136 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":20}
I20260806 22:51:30.501780 4817 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ada75c6549144c699dbed1844a24d91d: Generated new TSK 0
I20260806 22:51:30.502013 4817 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:51:30.504779 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 4706
2026-08-06T22:51:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1491 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056405724609-6305-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:51:30 dist-test-slave-zh2d krb5kdc[4846](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:51:30 dist-test-slave-zh2d krb5kdc[4846](info): set up 2 sockets
Aug 06 22:51:30 dist-test-slave-zh2d krb5kdc[4846](info): commencing operation
krb5kdc: starting...
W20260806 22:51:31.731462 6305 mini_kdc.cc:121] Time spent starting KDC: real 1.198s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056405724609-6305-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056405724609-6305-0/krb5kdc/test-user.keytab.
Aug 06 22:51:31 dist-test-slave-zh2d krb5kdc[4846](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056691, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:51:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:51:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.40.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.40.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.40.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab.
Entry for principal kudu/127.6.40.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab.
I20260806 22:51:31.857420 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056405724609-6305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056405724609-6305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056405724609-6305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.126:46257
--webserver_interface=127.6.40.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab
--principal=kudu/127.6.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:33499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.40.126:46257
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:51:31.941118 4862 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:51:31.941326 4862 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:51:31.941350 4862 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:51:31.942564 4862 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:51:31.942611 4862 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:51:31.942626 4862 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:51:31.942639 4862 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:51:31.942652 4862 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:51:31.944146 4862 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:33499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056405724609-6305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056405724609-6305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.40.126:46257
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab
--principal=kudu/127.6.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.126:46257
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4862
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056405724609-6305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:51:31.944404 4862 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:51:31.944649 4862 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:51:31.949844 4869 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:51:31.949844 4872 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:51:31.949880 4870 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:51:31.950078 4862 server_base.cc:1034] running on GCE node
I20260806 22:51:31.950344 4862 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:51:31.950667 4862 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:51:31.951860 4862 hybrid_clock.cc:648] HybridClock initialized: now 1786056691951812 us; error 61 us; skew 500 ppm
Aug 06 22:51:31 dist-test-slave-zh2d krb5kdc[4846](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056691, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:51:31.960168 4862 init.cc:377] Logged in from keytab as kudu/127.6.40.126@KRBTEST.COM (short username kudu)
I20260806 22:51:31.960585 4862 webserver.cc:460] Webserver started at http://127.6.40.126:41909/ using document root <none> and password file <none>
I20260806 22:51:31.960772 4862 fs_manager.cc:359] Metadata directory not provided
I20260806 22:51:31.960814 4862 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:51:31.960987 4862 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:51:31.962050 4862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056405724609-6305-0/minicluster-data/master-0/data/instance:
uuid: "d201ad53f21e4e5eba6ae5f536ee5125"
format_stamp: "Formatted at 2026-08-06 22:51:31 on dist-test-slave-zh2d"
server_key: "e71fce4a1acb4e28fee5479bce46db2b"
server_key_iv: "cbc7d9854b762e1db63af8c8168ea8a5"
server_key_version: "encryptionkey@0"
I20260806 22:51:31.962363 4862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056405724609-6305-0/minicluster-data/master-0/wal/instance:
uuid: "d201ad53f21e4e5eba6ae5f536ee5125"
format_stamp: "Formatted at 2026-08-06 22:51:31 on dist-test-slave-zh2d"
server_key: "e71fce4a1acb4e28fee5479bce46db2b"
server_key_iv: "cbc7d9854b762e1db63af8c8168ea8a5"
server_key_version: "encryptionkey@0"
I20260806 22:51:31.963680 4862 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:51:31.964385 4883 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:51:31.964550 4862 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:51:31.964641 4862 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056405724609-6305-0/minicluster-data/master-0/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056405724609-6305-0/minicluster-data/master-0/wal
uuid: "d201ad53f21e4e5eba6ae5f536ee5125"
format_stamp: "Formatted at 2026-08-06 22:51:31 on dist-test-slave-zh2d"
server_key: "e71fce4a1acb4e28fee5479bce46db2b"
server_key_iv: "cbc7d9854b762e1db63af8c8168ea8a5"
server_key_version: "encryptionkey@0"
I20260806 22:51:31.964699 4862 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056405724609-6305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056405724609-6305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056405724609-6305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:51:31.987700 4862 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:51:31.990243 4862 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:51:31.990386 4862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:51:31.995024 4862 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.126:46257
I20260806 22:51:31.995081 4965 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.126:46257 every 8 connection(s)
I20260806 22:51:31.995494 4862 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
I20260806 22:51:31.996424 4967 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:51:31.998762 4967 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d201ad53f21e4e5eba6ae5f536ee5125: Bootstrap starting.
I20260806 22:51:31.999361 4967 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d201ad53f21e4e5eba6ae5f536ee5125: Neither blocks nor log segments found. Creating new log.
I20260806 22:51:31.999680 4967 log.cc:824] T 00000000000000000000000000000000 P d201ad53f21e4e5eba6ae5f536ee5125: Log is configured to *not* fsync() on all Append() calls
I20260806 22:51:32.000473 4967 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d201ad53f21e4e5eba6ae5f536ee5125: No bootstrap required, opened a new log
I20260806 22:51:32.002470 4967 raft_consensus.cc:348] T 00000000000000000000000000000000 P d201ad53f21e4e5eba6ae5f536ee5125 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d201ad53f21e4e5eba6ae5f536ee5125" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 46257 } }
I20260806 22:51:32.002645 4967 raft_consensus.cc:374] T 00000000000000000000000000000000 P d201ad53f21e4e5eba6ae5f536ee5125 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:51:32.002692 4967 raft_consensus.cc:729] T 00000000000000000000000000000000 P d201ad53f21e4e5eba6ae5f536ee5125 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d201ad53f21e4e5eba6ae5f536ee5125, State: Initialized, Role: FOLLOWER
I20260806 22:51:32.002914 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 4862
I20260806 22:51:32.002905 4967 consensus_queue.cc:260] T 00000000000000000000000000000000 P d201ad53f21e4e5eba6ae5f536ee5125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d201ad53f21e4e5eba6ae5f536ee5125" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 46257 } }
I20260806 22:51:32.003000 4967 raft_consensus.cc:388] T 00000000000000000000000000000000 P d201ad53f21e4e5eba6ae5f536ee5125 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:51:32.003042 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056405724609-6305-0/minicluster-data/master-0/wal/instance
I20260806 22:51:32.003051 4967 raft_consensus.cc:482] T 00000000000000000000000000000000 P d201ad53f21e4e5eba6ae5f536ee5125 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:51:32.003113 4967 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d201ad53f21e4e5eba6ae5f536ee5125 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:51:32.003257 6305 external_mini_cluster.cc:1428] Setting key cd35e46030e16402d4cf6db1e46cf101
I20260806 22:51:32.004462 4967 raft_consensus.cc:504] T 00000000000000000000000000000000 P d201ad53f21e4e5eba6ae5f536ee5125 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d201ad53f21e4e5eba6ae5f536ee5125" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 46257 } }
I20260806 22:51:32.004580 4967 leader_election.cc:302] T 00000000000000000000000000000000 P d201ad53f21e4e5eba6ae5f536ee5125 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d201ad53f21e4e5eba6ae5f536ee5125; no voters:
I20260806 22:51:32.004755 4967 leader_election.cc:288] T 00000000000000000000000000000000 P d201ad53f21e4e5eba6ae5f536ee5125 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:51:32.004781 4974 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d201ad53f21e4e5eba6ae5f536ee5125 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:51:32.005039 4974 raft_consensus.cc:686] T 00000000000000000000000000000000 P d201ad53f21e4e5eba6ae5f536ee5125 [term 1 LEADER]: Becoming Leader. State: Replica: d201ad53f21e4e5eba6ae5f536ee5125, State: Running, Role: LEADER
I20260806 22:51:32.005185 4967 sys_catalog.cc:564] T 00000000000000000000000000000000 P d201ad53f21e4e5eba6ae5f536ee5125 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:51:32.005208 4974 consensus_queue.cc:237] T 00000000000000000000000000000000 P d201ad53f21e4e5eba6ae5f536ee5125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d201ad53f21e4e5eba6ae5f536ee5125" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 46257 } }
I20260806 22:51:32.005694 4975 sys_catalog.cc:455] T 00000000000000000000000000000000 P d201ad53f21e4e5eba6ae5f536ee5125 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d201ad53f21e4e5eba6ae5f536ee5125" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d201ad53f21e4e5eba6ae5f536ee5125" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 46257 } } }
I20260806 22:51:32.005810 4975 sys_catalog.cc:458] T 00000000000000000000000000000000 P d201ad53f21e4e5eba6ae5f536ee5125 [sys.catalog]: This master's current role is: LEADER
I20260806 22:51:32.006011 4976 sys_catalog.cc:455] T 00000000000000000000000000000000 P d201ad53f21e4e5eba6ae5f536ee5125 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d201ad53f21e4e5eba6ae5f536ee5125. Latest consensus state: current_term: 1 leader_uuid: "d201ad53f21e4e5eba6ae5f536ee5125" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d201ad53f21e4e5eba6ae5f536ee5125" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 46257 } } }
I20260806 22:51:32.006079 4976 sys_catalog.cc:458] T 00000000000000000000000000000000 P d201ad53f21e4e5eba6ae5f536ee5125 [sys.catalog]: This master's current role is: LEADER
I20260806 22:51:32.006407 4982 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:51:32.007033 4982 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:51:32.009232 4982 catalog_manager.cc:1347] Generated new cluster ID: 232d1825fc2044ee899b53d956219d19
I20260806 22:51:32.009294 4982 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:51:32.009995 4973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:32.003806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55633 (local address 127.6.40.126:46257)
0806 22:51:32.004063 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:32.004077 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:32.004124 (+ 47us) server_negotiation.cc:407] Connection header received
0806 22:51:32.004248 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:32.004255 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:32.004335 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:32.004510 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:32.004743 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:32.006162 (+ 1419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:32.007355 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:32.008013 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:32.008100 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:32.009005 (+ 905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:51:32.009010 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:51:32.009010 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:51:32.009242 (+ 232us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:51:32.009286 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:51:32.009303 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:51:32.009391 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:32.009690 (+ 299us) server_negotiation.cc:299] Negotiation successful
0806 22:51:32.009765 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":129,"threads_started":1}
I20260806 22:51:32.011003 6305 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:51:32.016535 4973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:32.012577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54273 (local address 127.6.40.126:46257)
0806 22:51:32.012636 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:32.012638 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:32.012762 (+ 124us) server_negotiation.cc:407] Connection header received
0806 22:51:32.012809 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:32.012810 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:32.012823 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:32.012928 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:32.013036 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:32.014094 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:32.015167 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:32.015468 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:32.015543 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:32.016269 (+ 726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:51:32.016270 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:51:32.016270 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:51:32.016275 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:51:32.016291 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:51:32.016295 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:51:32.016355 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:32.016435 (+ 80us) server_negotiation.cc:299] Negotiation successful
0806 22:51:32.016460 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 22:51:32.020635 4982 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:51:32.021257 4982 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:51:32.026235 4982 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d201ad53f21e4e5eba6ae5f536ee5125: Generated new TSK 0
I20260806 22:51:32.026438 4982 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:51:32.031306 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 4862
2026-08-06T22:51:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1528 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056405724609-6305-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:51:32 dist-test-slave-zh2d krb5kdc[5004](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:51:32 dist-test-slave-zh2d krb5kdc[5004](info): set up 2 sockets
Aug 06 22:51:32 dist-test-slave-zh2d krb5kdc[5004](info): commencing operation
krb5kdc: starting...
W20260806 22:51:33.370788 6305 mini_kdc.cc:121] Time spent starting KDC: real 1.309s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056405724609-6305-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056405724609-6305-0/krb5kdc/test-user.keytab.
Aug 06 22:51:33 dist-test-slave-zh2d krb5kdc[5004](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056693, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:51:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:51:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.40.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.40.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.40.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab.
Entry for principal kudu/127.6.40.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab.
I20260806 22:51:33.500036 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056405724609-6305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056405724609-6305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056405724609-6305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.126:46519
--webserver_interface=127.6.40.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab
--principal=kudu/127.6.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:45237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.40.126:46519
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:51:33.583006 5020 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:51:33.583212 5020 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:51:33.583235 5020 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:51:33.584331 5020 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:51:33.584371 5020 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:51:33.584386 5020 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:51:33.584398 5020 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:51:33.584410 5020 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:51:33.585826 5020 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:45237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056405724609-6305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056405724609-6305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.40.126:46519
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab
--principal=kudu/127.6.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.126:46519
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5020
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056405724609-6305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:51:33.586020 5020 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:51:33.586193 5020 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:51:33.591347 5027 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:51:33.591367 5026 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:51:33.591320 5020 server_base.cc:1034] running on GCE node
W20260806 22:51:33.591320 5029 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:51:33.591866 5020 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:51:33.592223 5020 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:51:33.593441 5020 hybrid_clock.cc:648] HybridClock initialized: now 1786056693593434 us; error 39 us; skew 500 ppm
Aug 06 22:51:33 dist-test-slave-zh2d krb5kdc[5004](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056693, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:51:33.596297 5020 init.cc:377] Logged in from keytab as kudu/127.6.40.126@KRBTEST.COM (short username kudu)
I20260806 22:51:33.596800 5020 webserver.cc:460] Webserver started at http://127.6.40.126:37965/ using document root <none> and password file <none>
I20260806 22:51:33.597108 5020 fs_manager.cc:359] Metadata directory not provided
I20260806 22:51:33.597179 5020 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:51:33.597337 5020 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:51:33.598438 5020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056405724609-6305-0/minicluster-data/master-0/data/instance:
uuid: "b1197ce35339403bbc4dbde601be0f59"
format_stamp: "Formatted at 2026-08-06 22:51:33 on dist-test-slave-zh2d"
server_key: "0493faeba6d16d814cd42cc4305fee04"
server_key_iv: "18cd7aa93b22da5e9709ea6375a27763"
server_key_version: "encryptionkey@0"
I20260806 22:51:33.598812 5020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056405724609-6305-0/minicluster-data/master-0/wal/instance:
uuid: "b1197ce35339403bbc4dbde601be0f59"
format_stamp: "Formatted at 2026-08-06 22:51:33 on dist-test-slave-zh2d"
server_key: "0493faeba6d16d814cd42cc4305fee04"
server_key_iv: "18cd7aa93b22da5e9709ea6375a27763"
server_key_version: "encryptionkey@0"
I20260806 22:51:33.600173 5020 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:51:33.600991 5041 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:51:33.601169 5020 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:51:33.601270 5020 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056405724609-6305-0/minicluster-data/master-0/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056405724609-6305-0/minicluster-data/master-0/wal
uuid: "b1197ce35339403bbc4dbde601be0f59"
format_stamp: "Formatted at 2026-08-06 22:51:33 on dist-test-slave-zh2d"
server_key: "0493faeba6d16d814cd42cc4305fee04"
server_key_iv: "18cd7aa93b22da5e9709ea6375a27763"
server_key_version: "encryptionkey@0"
I20260806 22:51:33.601356 5020 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056405724609-6305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056405724609-6305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056405724609-6305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:51:33.604664 5020 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:51:33.607023 5020 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:51:33.607182 5020 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:51:33.611251 5020 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.126:46519
I20260806 22:51:33.611294 5111 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.126:46519 every 8 connection(s)
I20260806 22:51:33.611642 5020 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
I20260806 22:51:33.612654 5113 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:51:33.614982 5113 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1197ce35339403bbc4dbde601be0f59: Bootstrap starting.
I20260806 22:51:33.615300 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 5020
I20260806 22:51:33.615427 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056405724609-6305-0/minicluster-data/master-0/wal/instance
I20260806 22:51:33.615631 6305 external_mini_cluster.cc:1428] Setting key 2eb9d0c18cfb47ab66fe06ee1a75c42e
I20260806 22:51:33.615728 5113 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b1197ce35339403bbc4dbde601be0f59: Neither blocks nor log segments found. Creating new log.
I20260806 22:51:33.616112 5113 log.cc:824] T 00000000000000000000000000000000 P b1197ce35339403bbc4dbde601be0f59: Log is configured to *not* fsync() on all Append() calls
I20260806 22:51:33.616977 5113 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1197ce35339403bbc4dbde601be0f59: No bootstrap required, opened a new log
I20260806 22:51:33.618237 5117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:33.616119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34351 (local address 127.6.40.126:46519)
0806 22:51:33.616322 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:33.616332 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:33.616371 (+ 39us) server_negotiation.cc:407] Connection header received
0806 22:51:33.616986 (+ 615us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:33.616995 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:33.617029 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:33.617184 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:33.617784 (+ 600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:51:33.617788 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:51:33.617788 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:51:33.617793 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:51:33.617844 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:51:33.617859 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:51:33.617959 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:33.617989 (+ 30us) server_negotiation.cc:299] Negotiation successful
0806 22:51:33.618028 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":81,"threads_started":1}
I20260806 22:51:33.619369 6305 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:51:33.619550 5113 raft_consensus.cc:348] T 00000000000000000000000000000000 P b1197ce35339403bbc4dbde601be0f59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1197ce35339403bbc4dbde601be0f59" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 46519 } }
I20260806 22:51:33.619704 5113 raft_consensus.cc:374] T 00000000000000000000000000000000 P b1197ce35339403bbc4dbde601be0f59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:51:33.619736 5113 raft_consensus.cc:729] T 00000000000000000000000000000000 P b1197ce35339403bbc4dbde601be0f59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1197ce35339403bbc4dbde601be0f59, State: Initialized, Role: FOLLOWER
I20260806 22:51:33.619861 5113 consensus_queue.cc:260] T 00000000000000000000000000000000 P b1197ce35339403bbc4dbde601be0f59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1197ce35339403bbc4dbde601be0f59" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 46519 } }
I20260806 22:51:33.619959 5113 raft_consensus.cc:388] T 00000000000000000000000000000000 P b1197ce35339403bbc4dbde601be0f59 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:51:33.620002 5113 raft_consensus.cc:482] T 00000000000000000000000000000000 P b1197ce35339403bbc4dbde601be0f59 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:51:33.620051 5113 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b1197ce35339403bbc4dbde601be0f59 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:51:33.620786 5113 raft_consensus.cc:504] T 00000000000000000000000000000000 P b1197ce35339403bbc4dbde601be0f59 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1197ce35339403bbc4dbde601be0f59" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 46519 } }
I20260806 22:51:33.620934 5113 leader_election.cc:302] T 00000000000000000000000000000000 P b1197ce35339403bbc4dbde601be0f59 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b1197ce35339403bbc4dbde601be0f59; no voters:
I20260806 22:51:33.621100 5113 leader_election.cc:288] T 00000000000000000000000000000000 P b1197ce35339403bbc4dbde601be0f59 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:51:33.621448 5113 sys_catalog.cc:564] T 00000000000000000000000000000000 P b1197ce35339403bbc4dbde601be0f59 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:51:33.622193 5117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:33.621188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40707 (local address 127.6.40.126:46519)
0806 22:51:33.621221 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:33.621223 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:33.621279 (+ 56us) server_negotiation.cc:407] Connection header received
0806 22:51:33.621372 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:33.621373 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:33.621377 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:33.621429 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:33.621970 (+ 541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:51:33.621971 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:51:33.621972 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:51:33.621976 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:51:33.621991 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:51:33.621995 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:51:33.622051 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:33.622116 (+ 65us) server_negotiation.cc:299] Negotiation successful
0806 22:51:33.622130 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 22:51:33.621137 5126 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b1197ce35339403bbc4dbde601be0f59 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:51:33.623039 5126 raft_consensus.cc:686] T 00000000000000000000000000000000 P b1197ce35339403bbc4dbde601be0f59 [term 1 LEADER]: Becoming Leader. State: Replica: b1197ce35339403bbc4dbde601be0f59, State: Running, Role: LEADER
I20260806 22:51:33.623165 5126 consensus_queue.cc:237] T 00000000000000000000000000000000 P b1197ce35339403bbc4dbde601be0f59 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1197ce35339403bbc4dbde601be0f59" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 46519 } }
I20260806 22:51:33.623610 5141 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1197ce35339403bbc4dbde601be0f59 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b1197ce35339403bbc4dbde601be0f59. Latest consensus state: current_term: 1 leader_uuid: "b1197ce35339403bbc4dbde601be0f59" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1197ce35339403bbc4dbde601be0f59" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 46519 } } }
I20260806 22:51:33.623718 5128 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1197ce35339403bbc4dbde601be0f59 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b1197ce35339403bbc4dbde601be0f59" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1197ce35339403bbc4dbde601be0f59" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 46519 } } }
I20260806 22:51:33.623981 5128 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1197ce35339403bbc4dbde601be0f59 [sys.catalog]: This master's current role is: LEADER
I20260806 22:51:33.623837 5141 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1197ce35339403bbc4dbde601be0f59 [sys.catalog]: This master's current role is: LEADER
W20260806 22:51:33.624171 5143 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b1197ce35339403bbc4dbde601be0f59: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:51:33.624220 5143 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:51:33.624454 5146 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:51:33.624629 5146 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:51:33.626047 5146 catalog_manager.cc:1347] Generated new cluster ID: 154743370e4648e7bff3c7560b9bb476
I20260806 22:51:33.626119 5146 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:51:33.638450 5146 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:51:33.638630 5146 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:51:33.642886 5146 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b1197ce35339403bbc4dbde601be0f59: Generated new TSK 0
I20260806 22:51:33.643054 5146 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:51:33.648097 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 5020
2026-08-06T22:51:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1615 ms)
[----------] 18 tests from AuthTokenIssuingTest (27568 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056405724609-6305-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:51:33 dist-test-slave-zh2d krb5kdc[5150](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:51:33 dist-test-slave-zh2d krb5kdc[5150](info): set up 2 sockets
Aug 06 22:51:33 dist-test-slave-zh2d krb5kdc[5150](info): commencing operation
krb5kdc: starting...
W20260806 22:51:34.859542 6305 mini_kdc.cc:121] Time spent starting KDC: real 1.182s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056405724609-6305-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056405724609-6305-0/krb5kdc/test-user.keytab.
Aug 06 22:51:34 dist-test-slave-zh2d krb5kdc[5150](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056694, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:51:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:51:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.40.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.40.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.40.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab.
Entry for principal kudu/127.6.40.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab.
I20260806 22:51:34.987226 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056405724609-6305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056405724609-6305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056405724609-6305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.126:38461
--webserver_interface=127.6.40.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab
--principal=kudu/127.6.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:42675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.40.126:38461,127.6.40.125:38863,127.6.40.124:39017
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:51:35.069996 5166 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 22:51:35.070202 5166 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:51:35.070220 5166 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:51:35.070268 5166 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:51:35.071444 5166 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:51:35.071522 5166 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:51:35.071552 5166 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:51:35.071595 5166 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:51:35.071614 5166 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:51:35.073184 5166 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:42675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056405724609-6305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056405724609-6305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.40.126:38461,127.6.40.125:38863,127.6.40.124:39017
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab
--principal=kudu/127.6.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.126:38461
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5166
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056405724609-6305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:51:35.073433 5166 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:51:35.073664 5166 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:51:35.078528 5179 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:51:35.078526 5176 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:51:35.078526 5174 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:51:35.078653 5166 server_base.cc:1034] running on GCE node
I20260806 22:51:35.079026 5166 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:51:35.079358 5166 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:51:35.080543 5166 hybrid_clock.cc:648] HybridClock initialized: now 1786056695080519 us; error 42 us; skew 500 ppm
Aug 06 22:51:35 dist-test-slave-zh2d krb5kdc[5150](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056695, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:51:35.083535 5166 init.cc:377] Logged in from keytab as kudu/127.6.40.126@KRBTEST.COM (short username kudu)
I20260806 22:51:35.084086 5166 webserver.cc:460] Webserver started at http://127.6.40.126:33279/ using document root <none> and password file <none>
I20260806 22:51:35.084342 5166 fs_manager.cc:359] Metadata directory not provided
I20260806 22:51:35.084393 5166 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:51:35.084595 5166 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:51:35.085880 5166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056405724609-6305-0/minicluster-data/master-0/data/instance:
uuid: "a9be425d006c470d891f1f95f373b003"
format_stamp: "Formatted at 2026-08-06 22:51:35 on dist-test-slave-zh2d"
server_key: "15b6309e676d798d40d6b5101b57364c"
server_key_iv: "c2648d2c55fa93fadc7771955f4e6122"
server_key_version: "encryptionkey@0"
I20260806 22:51:35.086297 5166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056405724609-6305-0/minicluster-data/master-0/wal/instance:
uuid: "a9be425d006c470d891f1f95f373b003"
format_stamp: "Formatted at 2026-08-06 22:51:35 on dist-test-slave-zh2d"
server_key: "15b6309e676d798d40d6b5101b57364c"
server_key_iv: "c2648d2c55fa93fadc7771955f4e6122"
server_key_version: "encryptionkey@0"
I20260806 22:51:35.087755 5166 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:51:35.088565 5188 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:51:35.088747 5166 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:51:35.088915 5166 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056405724609-6305-0/minicluster-data/master-0/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056405724609-6305-0/minicluster-data/master-0/wal
uuid: "a9be425d006c470d891f1f95f373b003"
format_stamp: "Formatted at 2026-08-06 22:51:35 on dist-test-slave-zh2d"
server_key: "15b6309e676d798d40d6b5101b57364c"
server_key_iv: "c2648d2c55fa93fadc7771955f4e6122"
server_key_version: "encryptionkey@0"
I20260806 22:51:35.089004 5166 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056405724609-6305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056405724609-6305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056405724609-6305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:51:35.107623 5166 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:51:35.110328 5166 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:51:35.110522 5166 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:51:35.115247 5166 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.126:38461
I20260806 22:51:35.115283 5255 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.126:38461 every 8 connection(s)
I20260806 22:51:35.115669 5166 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
I20260806 22:51:35.116701 5256 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:51:35.118297 5256 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.40.126" port: 38461 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:51:35.122421 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 5166
I20260806 22:51:35.122553 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056405724609-6305-0/minicluster-data/master-0/wal/instance
I20260806 22:51:35.122751 6305 external_mini_cluster.cc:1428] Setting key 3f9c1ab44d4753a76afc9f3a317d1c66
Aug 06 22:51:35 dist-test-slave-zh2d krb5kdc[5150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056695, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.126@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
I20260806 22:51:35.130045 5259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:35.118961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:38461 (local address 127.0.0.1:45508)
0806 22:51:35.119185 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0806 22:51:35.119228 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 22:51:35.119720 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:51:35.120035 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:51:35.120038 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:51:35.120303 (+ 265us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:51:35.120464 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:35.120468 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:35.121664 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:35.121665 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:35.122471 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:35.122474 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:35.123117 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:35.123118 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:35.123203 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:35.123721 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:51:35.123731 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:51:35.125908 (+ 2177us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:51:35.128347 (+ 2439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:51:35.128354 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:51:35.128355 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:51:35.128568 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:51:35.128898 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:51:35.128898 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:51:35.128899 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:51:35.129005 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:51:35.129498 (+ 493us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:51:35.129504 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:51:35.129563 (+ 59us) client_negotiation.cc:769] Sending connection context
0806 22:51:35.129634 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 22:51:35.129762 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":99,"threads_started":1}
I20260806 22:51:35.130052 5261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:35.119002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45508 (local address 127.6.40.126:38461)
0806 22:51:35.119194 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:35.119203 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:35.119329 (+ 126us) server_negotiation.cc:407] Connection header received
0806 22:51:35.119885 (+ 556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:35.119888 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:35.119908 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:35.120020 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:35.120568 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:35.121556 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:35.122581 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:35.122997 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:35.123097 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:35.126012 (+ 2915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:51:35.126019 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:51:35.126019 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:51:35.126026 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:51:35.128212 (+ 2186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:35.128659 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:35.128663 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:35.128663 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:35.128776 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:35.129080 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:35.129081 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:35.129082 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:35.129410 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:51:35.129518 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:35.129705 (+ 187us) server_negotiation.cc:299] Negotiation successful
0806 22:51:35.129862 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":30,"server-negotiator.queue_time_us":150,"thread_start_us":88,"threads_started":1}
I20260806 22:51:35.130923 5256 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.40.125" port: 38863 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:51:35.131287 5259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:35.131175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.125:38863 (local address 127.0.0.1:42326)
0806 22:51:35.131224 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0806 22:51:35.131268 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.125:38863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260806 22:51:35.131507 5193 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.40.125:38863: connect: Connection refused (error 111)
I20260806 22:51:35.131956 5259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:35.131864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.125:38863 (local address 127.0.0.1:42332)
0806 22:51:35.131915 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0806 22:51:35.131940 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.125:38863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260806 22:51:35.132097 5256 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.40.125:38863: Network error: Client connection negotiation failed: client connection to 127.6.40.125:38863: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.6.40.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.40.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.40.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.125.keytab.
Entry for principal kudu/127.6.40.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.125.keytab.
I20260806 22:51:35.149950 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056405724609-6305-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056405724609-6305-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056405724609-6305-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056405724609-6305-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.125:38863
--webserver_interface=127.6.40.125
--webserver_port=0
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.125.keytab
--principal=kudu/127.6.40.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:42675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.40.126:38461,127.6.40.125:38863,127.6.40.124:39017
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:51:35.181347 5256 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.40.125" port: 38863 } attempt: 1
I20260806 22:51:35.181915 5259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:35.181755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.125:38863 (local address 127.0.0.1:42338)
0806 22:51:35.181826 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0806 22:51:35.181876 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.125:38863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260806 22:51:35.182350 5259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:35.182252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.125:38863 (local address 127.0.0.1:42344)
0806 22:51:35.182297 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0806 22:51:35.182327 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.125:38863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260806 22:51:35.182466 5256 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.40.125:38863: Network error: Client connection negotiation failed: client connection to 127.6.40.125:38863: connect: Connection refused (error 111)
W20260806 22:51:35.233829 5266 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 22:51:35.234047 5266 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:51:35.234066 5266 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:51:35.234086 5266 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:51:35.235131 5266 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:51:35.235163 5266 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:51:35.235177 5266 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:51:35.235189 5266 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:51:35.235201 5266 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:51:35.236618 5266 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:42675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056405724609-6305-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056405724609-6305-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.6.40.126:38461,127.6.40.125:38863,127.6.40.124:39017
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.125.keytab
--principal=kudu/127.6.40.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.125:38863
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056405724609-6305-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.6.40.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5266
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056405724609-6305-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:51:35.236810 5266 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:51:35.237067 5266 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:51:35.242532 5274 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:51:35.242560 5273 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:51:35.242594 5266 server_base.cc:1034] running on GCE node
W20260806 22:51:35.242561 5278 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:51:35.243083 5266 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:51:35.243384 5266 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:51:35.244554 5266 hybrid_clock.cc:648] HybridClock initialized: now 1786056695244536 us; error 42 us; skew 500 ppm
Aug 06 22:51:35 dist-test-slave-zh2d krb5kdc[5150](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056695, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:51:35.247252 5266 init.cc:377] Logged in from keytab as kudu/127.6.40.125@KRBTEST.COM (short username kudu)
I20260806 22:51:35.247788 5266 webserver.cc:460] Webserver started at http://127.6.40.125:42533/ using document root <none> and password file <none>
I20260806 22:51:35.248040 5266 fs_manager.cc:359] Metadata directory not provided
I20260806 22:51:35.248111 5266 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:51:35.248250 5266 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:51:35.249382 5266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056405724609-6305-0/minicluster-data/master-1/data/instance:
uuid: "78ab2695431f473198a5fcbbf4a71642"
format_stamp: "Formatted at 2026-08-06 22:51:35 on dist-test-slave-zh2d"
server_key: "76c8a2854264210d63da01e24851d35c"
server_key_iv: "4a1e7745f1f14fe38b20540bbdad907f"
server_key_version: "encryptionkey@0"
I20260806 22:51:35.249759 5266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056405724609-6305-0/minicluster-data/master-1/wal/instance:
uuid: "78ab2695431f473198a5fcbbf4a71642"
format_stamp: "Formatted at 2026-08-06 22:51:35 on dist-test-slave-zh2d"
server_key: "76c8a2854264210d63da01e24851d35c"
server_key_iv: "4a1e7745f1f14fe38b20540bbdad907f"
server_key_version: "encryptionkey@0"
I20260806 22:51:35.250613 5256 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.40.125" port: 38863 } attempt: 2
I20260806 22:51:35.251037 5259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:35.250911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.125:38863 (local address 127.0.0.1:42346)
0806 22:51:35.250975 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0806 22:51:35.251010 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.125:38863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260806 22:51:35.251319 5266 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:51:35.251485 5259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:35.251377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.125:38863 (local address 127.0.0.1:42358)
0806 22:51:35.251430 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0806 22:51:35.251468 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.125:38863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260806 22:51:35.251600 5256 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.40.125:38863: Network error: Client connection negotiation failed: client connection to 127.6.40.125:38863: connect: Connection refused (error 111)
I20260806 22:51:35.252187 5288 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:51:35.252388 5266 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:51:35.252481 5266 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056405724609-6305-0/minicluster-data/master-1/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056405724609-6305-0/minicluster-data/master-1/wal
uuid: "78ab2695431f473198a5fcbbf4a71642"
format_stamp: "Formatted at 2026-08-06 22:51:35 on dist-test-slave-zh2d"
server_key: "76c8a2854264210d63da01e24851d35c"
server_key_iv: "4a1e7745f1f14fe38b20540bbdad907f"
server_key_version: "encryptionkey@0"
I20260806 22:51:35.252535 5266 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056405724609-6305-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056405724609-6305-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056405724609-6305-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:51:35.266467 5266 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:51:35.268920 5266 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:51:35.269038 5266 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:51:35.273105 5266 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.125:38863
I20260806 22:51:35.273206 5369 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.125:38863 every 8 connection(s)
I20260806 22:51:35.273520 5266 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056405724609-6305-0/minicluster-data/master-1/data/info.pb
I20260806 22:51:35.274554 5370 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:51:35.275221 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 5266
I20260806 22:51:35.275305 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056405724609-6305-0/minicluster-data/master-1/wal/instance
I20260806 22:51:35.275542 6305 external_mini_cluster.cc:1428] Setting key 5ce288af684e0b2749f02bc8627bf976
I20260806 22:51:35.276468 5370 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.40.126" port: 38461 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 22:51:35 dist-test-slave-zh2d krb5kdc[5150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056695, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.125@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.40.124@KRBTEST.COM; defaulting to no policy
I20260806 22:51:35.286690 5261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:35.277581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45512 (local address 127.6.40.126:38461)
0806 22:51:35.277656 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:35.277658 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:35.277782 (+ 124us) server_negotiation.cc:407] Connection header received
0806 22:51:35.278256 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:35.278257 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:35.278275 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:35.278355 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:35.278984 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:35.279515 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:35.281140 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:35.281357 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:35.281415 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:35.283945 (+ 2530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:51:35.283947 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:51:35.283947 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:51:35.283956 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:51:35.285510 (+ 1554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:35.285866 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:35.285867 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:35.285867 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:35.285925 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:35.286182 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:35.286183 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:35.286183 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:35.286282 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:51:35.286332 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:35.286559 (+ 227us) server_negotiation.cc:299] Negotiation successful
0806 22:51:35.286624 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 22:51:35.286881 5373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:35.277236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:38461 (local address 127.0.0.1:45512)
0806 22:51:35.277664 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0806 22:51:35.277699 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 22:51:35.278162 (+ 463us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:51:35.278427 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:51:35.278430 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:51:35.278674 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:51:35.278898 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:35.278902 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:35.279621 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:35.279622 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:35.281058 (+ 1436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:35.281061 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:35.281473 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:35.281475 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:35.281635 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:35.282165 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:51:35.282177 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:51:35.283844 (+ 1667us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:51:35.285586 (+ 1742us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:51:35.285593 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:51:35.285595 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:51:35.285799 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:51:35.285984 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:51:35.285984 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:51:35.285985 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:51:35.286080 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:51:35.286327 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:51:35.286331 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:51:35.286448 (+ 117us) client_negotiation.cc:769] Sending connection context
0806 22:51:35.286531 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 22:51:35.286649 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":83,"threads_started":1}
I20260806 22:51:35.287631 5370 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.40.125" port: 38863 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.40.124@KRBTEST.COM" created.
Aug 06 22:51:35 dist-test-slave-zh2d krb5kdc[5150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056695, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.125@KRBTEST.COM for kudu/127.6.40.125@KRBTEST.COM
I20260806 22:51:35.296170 5373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:35.287988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.125:38863 (local address 127.0.0.1:42374)
0806 22:51:35.288050 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0806 22:51:35.288062 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:51:35.288111 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:51:35.288360 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:51:35.288361 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:51:35.288502 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:51:35.288550 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:35.288551 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:35.289848 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:35.289849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:35.290697 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:35.290701 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:35.291171 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:35.291172 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:35.291256 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:35.291750 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:51:35.291756 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:51:35.293148 (+ 1392us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:51:35.295041 (+ 1893us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:51:35.295042 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:51:35.295042 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:51:35.295212 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:51:35.295483 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:51:35.295484 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:51:35.295484 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:51:35.295542 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:51:35.295845 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:51:35.295846 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:51:35.295886 (+ 40us) client_negotiation.cc:769] Sending connection context
0806 22:51:35.295952 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 22:51:35.296057 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260806 22:51:35.296339 5374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:35.288045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42374 (local address 127.6.40.125:38863)
0806 22:51:35.288206 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:35.288210 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:35.288230 (+ 20us) server_negotiation.cc:407] Connection header received
0806 22:51:35.288259 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:35.288260 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:35.288271 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:35.288334 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:35.288628 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:35.289736 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:35.290784 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:35.291088 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:35.291145 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:35.293259 (+ 2114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:51:35.293269 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:51:35.293269 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:51:35.293276 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:51:35.294930 (+ 1654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:35.295304 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:35.295307 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:35.295307 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:35.295401 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:35.295606 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:35.295607 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:35.295607 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:35.295760 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:51:35.295850 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:35.295993 (+ 143us) server_negotiation.cc:299] Negotiation successful
0806 22:51:35.296115 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":48,"server-negotiator.queue_time_us":131,"thread_start_us":101,"threads_started":1}
I20260806 22:51:35.296988 5370 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.40.124" port: 39017 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:51:35.297365 5373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:35.297239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.124:39017 (local address 127.0.0.1:52266)
0806 22:51:35.297290 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0806 22:51:35.297339 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.124:39017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260806 22:51:35.297523 5295 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.40.124:39017: connect: Connection refused (error 111)
I20260806 22:51:35.298000 5373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:35.297906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.124:39017 (local address 127.0.0.1:52280)
0806 22:51:35.297947 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0806 22:51:35.297973 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.124:39017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260806 22:51:35.298127 5370 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.40.124:39017: Network error: Client connection negotiation failed: client connection to 127.6.40.124:39017: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.40.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.124.keytab.
Entry for principal kudu/127.6.40.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.124.keytab.
I20260806 22:51:35.304191 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056405724609-6305-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056405724609-6305-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056405724609-6305-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056405724609-6305-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.124:39017
--webserver_interface=127.6.40.124
--webserver_port=0
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.124.keytab
--principal=kudu/127.6.40.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:42675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.40.126:38461,127.6.40.125:38863,127.6.40.124:39017
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:51:35.342772 5256 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.40.125" port: 38863 } attempt: 3
Aug 06 22:51:35 dist-test-slave-zh2d krb5kdc[5150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056695, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.126@KRBTEST.COM for kudu/127.6.40.125@KRBTEST.COM
I20260806 22:51:35.347368 5370 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.40.124" port: 39017 } attempt: 1
I20260806 22:51:35.347761 5373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:35.347619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.124:39017 (local address 127.0.0.1:52294)
0806 22:51:35.347685 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0806 22:51:35.347719 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.124:39017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260806 22:51:35.348121 5373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:35.348009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.124:39017 (local address 127.0.0.1:52310)
0806 22:51:35.348062 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0806 22:51:35.348091 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.124:39017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260806 22:51:35.348263 5370 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.40.124:39017: Network error: Client connection negotiation failed: client connection to 127.6.40.124:39017: connect: Connection refused (error 111)
I20260806 22:51:35.350382 5374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:35.343304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42386 (local address 127.6.40.125:38863)
0806 22:51:35.343370 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:35.343373 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:35.343384 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:51:35.343434 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:35.343435 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:35.343447 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:35.343524 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:35.343863 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:35.344377 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:35.345197 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:35.345430 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:35.345512 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:35.347645 (+ 2133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:51:35.347647 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:51:35.347647 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:51:35.347655 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:51:35.349234 (+ 1579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:35.349602 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:35.349603 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:35.349604 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:35.349661 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:35.349863 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:35.349863 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:35.349864 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:35.349979 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:51:35.350068 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:35.350230 (+ 162us) server_negotiation.cc:299] Negotiation successful
0806 22:51:35.350301 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 22:51:35.350389 5259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:35.343233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.125:38863 (local address 127.0.0.1:42386)
0806 22:51:35.343298 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0806 22:51:35.343308 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:51:35.343370 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:51:35.343533 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:51:35.343534 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:51:35.343711 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:51:35.343776 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:35.343778 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:35.344476 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:35.344477 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:35.345110 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:35.345113 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:35.345525 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:35.345526 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:35.345605 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:35.346099 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:51:35.346103 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:51:35.347551 (+ 1448us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:51:35.349332 (+ 1781us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:51:35.349333 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:51:35.349333 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:51:35.349514 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:51:35.349738 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:51:35.349738 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:51:35.349739 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:51:35.349801 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:51:35.350060 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:51:35.350061 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:51:35.350109 (+ 48us) client_negotiation.cc:769] Sending connection context
0806 22:51:35.350192 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 22:51:35.350292 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260806 22:51:35.350873 5256 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.40.124" port: 39017 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:51:35.351214 5259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:35.351101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.124:39017 (local address 127.0.0.1:52324)
0806 22:51:35.351160 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0806 22:51:35.351186 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.124:39017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260806 22:51:35.351541 5259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:35.351485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.124:39017 (local address 127.0.0.1:52326)
0806 22:51:35.351508 (+ 23us) negotiation.cc:107] Waiting for socket to connect
0806 22:51:35.351529 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.124:39017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
W20260806 22:51:35.351645 5256 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.40.124:39017: Network error: Client connection negotiation failed: client connection to 127.6.40.124:39017: connect: Connection refused (error 111)
I20260806 22:51:35.382864 5256 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.40.124" port: 39017 } attempt: 1
I20260806 22:51:35.383407 5259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:35.383266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.124:39017 (local address 127.0.0.1:36896)
0806 22:51:35.383337 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0806 22:51:35.383379 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.124:39017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260806 22:51:35.383805 5259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:35.383720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.124:39017 (local address 127.0.0.1:36904)
0806 22:51:35.383762 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0806 22:51:35.383787 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.124:39017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260806 22:51:35.383930 5256 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.40.124:39017: Network error: Client connection negotiation failed: client connection to 127.6.40.124:39017: connect: Connection refused (error 111)
W20260806 22:51:35.390179 5377 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 22:51:35.390389 5377 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:51:35.390407 5377 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:51:35.390427 5377 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:51:35.391563 5377 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:51:35.391605 5377 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:51:35.391619 5377 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:51:35.391633 5377 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:51:35.391644 5377 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:51:35.393352 5377 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:42675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056405724609-6305-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056405724609-6305-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.6.40.126:38461,127.6.40.125:38863,127.6.40.124:39017
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.124.keytab
--principal=kudu/127.6.40.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.124:39017
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056405724609-6305-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.6.40.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5377
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056405724609-6305-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:51:35.393558 5377 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:51:35.393735 5377 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:51:35.398715 5386 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:51:35.398721 5384 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:51:35.398818 5388 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:51:35.398970 5377 server_base.cc:1034] running on GCE node
I20260806 22:51:35.399263 5377 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:51:35.399569 5377 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:51:35.400779 5377 hybrid_clock.cc:648] HybridClock initialized: now 1786056695400748 us; error 35 us; skew 500 ppm
Aug 06 22:51:35 dist-test-slave-zh2d krb5kdc[5150](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056695, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:51:35.403409 5377 init.cc:377] Logged in from keytab as kudu/127.6.40.124@KRBTEST.COM (short username kudu)
I20260806 22:51:35.403872 5377 webserver.cc:460] Webserver started at http://127.6.40.124:46121/ using document root <none> and password file <none>
I20260806 22:51:35.404062 5377 fs_manager.cc:359] Metadata directory not provided
I20260806 22:51:35.404103 5377 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:51:35.404192 5377 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:51:35.405289 5377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056405724609-6305-0/minicluster-data/master-2/data/instance:
uuid: "de4cafdc09a84f2e98fc738b9df0fce5"
format_stamp: "Formatted at 2026-08-06 22:51:35 on dist-test-slave-zh2d"
server_key: "f3fb01aae85c84907ca2ca768b3b0363"
server_key_iv: "14de605ca2457b5a71de054de33fdf56"
server_key_version: "encryptionkey@0"
I20260806 22:51:35.405604 5377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056405724609-6305-0/minicluster-data/master-2/wal/instance:
uuid: "de4cafdc09a84f2e98fc738b9df0fce5"
format_stamp: "Formatted at 2026-08-06 22:51:35 on dist-test-slave-zh2d"
server_key: "f3fb01aae85c84907ca2ca768b3b0363"
server_key_iv: "14de605ca2457b5a71de054de33fdf56"
server_key_version: "encryptionkey@0"
I20260806 22:51:35.406922 5377 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:51:35.407713 5397 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:51:35.407935 5377 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:51:35.408030 5377 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056405724609-6305-0/minicluster-data/master-2/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056405724609-6305-0/minicluster-data/master-2/wal
uuid: "de4cafdc09a84f2e98fc738b9df0fce5"
format_stamp: "Formatted at 2026-08-06 22:51:35 on dist-test-slave-zh2d"
server_key: "f3fb01aae85c84907ca2ca768b3b0363"
server_key_iv: "14de605ca2457b5a71de054de33fdf56"
server_key_version: "encryptionkey@0"
I20260806 22:51:35.408087 5377 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056405724609-6305-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056405724609-6305-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056405724609-6305-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:51:35.416432 5370 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.40.124" port: 39017 } attempt: 2
I20260806 22:51:35.416931 5373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:35.416734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.124:39017 (local address 127.0.0.1:36918)
0806 22:51:35.416796 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0806 22:51:35.416887 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.124:39017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260806 22:51:35.417321 5373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:35.417229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.124:39017 (local address 127.0.0.1:36920)
0806 22:51:35.417276 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0806 22:51:35.417303 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.124:39017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260806 22:51:35.417441 5370 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.40.124:39017: Network error: Client connection negotiation failed: client connection to 127.6.40.124:39017: connect: Connection refused (error 111)
I20260806 22:51:35.428684 5377 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:51:35.430660 5377 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:51:35.430774 5377 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:51:35.434840 5377 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.124:39017
I20260806 22:51:35.434912 5468 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.124:39017 every 8 connection(s)
I20260806 22:51:35.435262 5377 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056405724609-6305-0/minicluster-data/master-2/data/info.pb
I20260806 22:51:35.436318 5469 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:51:35.437999 5469 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.40.126" port: 38461 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:51:35.439533 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 5377
I20260806 22:51:35.439639 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056405724609-6305-0/minicluster-data/master-2/wal/instance
I20260806 22:51:35.439822 6305 external_mini_cluster.cc:1428] Setting key d9d12b80c276aeba5688e05ca1112949
Aug 06 22:51:35 dist-test-slave-zh2d krb5kdc[5150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056695, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.124@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
Aug 06 22:51:35 dist-test-slave-zh2d krb5kdc[5150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056694, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.40.124@KRBTEST.COM
I20260806 22:51:35.448740 5261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:35.438804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59464 (local address 127.6.40.126:38461)
0806 22:51:35.438875 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:35.438878 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:35.439097 (+ 219us) server_negotiation.cc:407] Connection header received
0806 22:51:35.439599 (+ 502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:35.439601 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:35.439613 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:35.439665 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:35.440562 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:35.441363 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:35.442749 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:35.443031 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:35.443103 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:35.445447 (+ 2344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:51:35.445449 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:51:35.445449 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:51:35.445457 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:51:35.447084 (+ 1627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:35.447637 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:35.447638 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:35.447638 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:35.447722 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:35.447978 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:35.447979 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:35.447987 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:35.448133 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:51:35.448213 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:35.448584 (+ 371us) server_negotiation.cc:299] Negotiation successful
0806 22:51:35.448668 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 22:51:35.449059 5472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:35.438791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:38461 (local address 127.0.0.1:59464)
0806 22:51:35.438977 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0806 22:51:35.439015 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 22:51:35.439476 (+ 461us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:51:35.439735 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:51:35.439738 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:51:35.440143 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:51:35.440459 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:35.440466 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:35.441453 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:35.441454 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:35.442660 (+ 1206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:35.442665 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:35.443130 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:35.443131 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:35.443228 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:35.443780 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:51:35.443790 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:51:35.445348 (+ 1558us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:51:35.447186 (+ 1838us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:51:35.447193 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:51:35.447196 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:51:35.447557 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:51:35.447796 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:51:35.447797 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:51:35.447797 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:51:35.447906 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:51:35.448238 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:51:35.448245 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:51:35.448428 (+ 183us) client_negotiation.cc:769] Sending connection context
0806 22:51:35.448537 (+ 109us) client_negotiation.cc:240] Negotiation successful
0806 22:51:35.448752 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"mutex_wait_us":39,"thread_start_us":104,"threads_started":1}
I20260806 22:51:35.449740 5469 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.40.125" port: 38863 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:51:35.450227 5474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:35.440253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36922 (local address 127.6.40.124:39017)
0806 22:51:35.440436 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:35.440440 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:35.440458 (+ 18us) server_negotiation.cc:407] Connection header received
0806 22:51:35.440511 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:35.440515 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:35.440535 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:35.440613 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:35.441453 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:35.442731 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:35.443654 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:35.444102 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:35.444179 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:35.446747 (+ 2568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:51:35.446756 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:51:35.446756 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:51:35.446764 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:51:35.448502 (+ 1738us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:35.449015 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:35.449019 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:35.449019 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:35.449080 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:35.449337 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:35.449338 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:35.449339 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:35.449514 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:51:35.449651 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:35.449957 (+ 306us) server_negotiation.cc:299] Negotiation successful
0806 22:51:35.450059 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":36,"server-negotiator.queue_time_us":158,"thread_start_us":97,"threads_started":1}
Aug 06 22:51:35 dist-test-slave-zh2d krb5kdc[5150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056695, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.124@KRBTEST.COM for kudu/127.6.40.125@KRBTEST.COM
Aug 06 22:51:35 dist-test-slave-zh2d krb5kdc[5150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056694, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
I20260806 22:51:35.457701 5472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:35.450066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.125:38863 (local address 127.0.0.1:57462)
0806 22:51:35.450121 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0806 22:51:35.450133 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:51:35.450177 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:51:35.450365 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:51:35.450366 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:51:35.450783 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:51:35.450885 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:35.450887 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:35.451705 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:35.451706 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:35.452269 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:35.452272 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:35.452715 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:35.452716 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:35.452797 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:35.453188 (+ 391us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:51:35.453194 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:51:35.454521 (+ 1327us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:51:35.456487 (+ 1966us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:51:35.456488 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:51:35.456489 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:51:35.456732 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:51:35.457052 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:51:35.457053 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:51:35.457053 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:51:35.457128 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:51:35.457475 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:51:35.457476 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:51:35.457513 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 22:51:35.457566 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 22:51:35.457640 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260806 22:51:35.457808 5374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:35.450130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57462 (local address 127.6.40.125:38863)
0806 22:51:35.450190 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:35.450192 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:35.450203 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:51:35.450234 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:35.450235 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:35.450246 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:35.450305 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:35.450996 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:35.451611 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:35.452412 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:35.452623 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:35.452677 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:35.454610 (+ 1933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:51:35.454613 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:51:35.454613 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:51:35.454622 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:51:35.456369 (+ 1747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:35.456827 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:35.456828 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:35.456828 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:35.456949 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:35.457202 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:35.457204 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:35.457204 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:35.457396 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:51:35.457488 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:35.457617 (+ 129us) server_negotiation.cc:299] Negotiation successful
0806 22:51:35.457696 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 22:51:35.458184 5469 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.40.124" port: 39017 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:51:35.459106 5256 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.40.124" port: 39017 } attempt: 2
I20260806 22:51:35.459256 5261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:35.451106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59466 (local address 127.6.40.126:38461)
0806 22:51:35.451162 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:35.451164 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:35.451171 (+ 7us) server_negotiation.cc:407] Connection header received
0806 22:51:35.451191 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:35.451191 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:35.451200 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:35.451261 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:35.451780 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:35.452217 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:35.453266 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:35.453440 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:35.453496 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:35.455954 (+ 2458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:51:35.455955 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:51:35.455956 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:51:35.455963 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:51:35.457935 (+ 1972us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:35.458375 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:35.458376 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:35.458376 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:35.458447 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:35.458671 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:35.458672 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:35.458673 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:35.458787 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:51:35.458982 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:35.459020 (+ 38us) server_negotiation.cc:299] Negotiation successful
0806 22:51:35.459105 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
Aug 06 22:51:35 dist-test-slave-zh2d krb5kdc[5150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056695, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.124@KRBTEST.COM for kudu/127.6.40.124@KRBTEST.COM
Aug 06 22:51:35 dist-test-slave-zh2d krb5kdc[5150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056695, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.126@KRBTEST.COM for kudu/127.6.40.124@KRBTEST.COM
Aug 06 22:51:35 dist-test-slave-zh2d krb5kdc[5150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056694, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.40.125@KRBTEST.COM
I20260806 22:51:35.467258 5472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:35.458441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.124:39017 (local address 127.0.0.1:36934)
0806 22:51:35.458495 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0806 22:51:35.458505 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:51:35.458543 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:51:35.458647 (+ 104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:51:35.458648 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:51:35.458779 (+ 131us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:51:35.458805 (+ 26us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:35.458806 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:35.459610 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:35.459611 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:35.460132 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:35.460135 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:35.460575 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:35.460576 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:35.460623 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:35.461113 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:51:35.461118 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:51:35.462383 (+ 1265us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:51:35.466078 (+ 3695us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:51:35.466079 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:51:35.466079 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:51:35.466304 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:51:35.466543 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:51:35.466544 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:51:35.466545 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:51:35.466613 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:51:35.466883 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:51:35.466884 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:51:35.466922 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 22:51:35.466990 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 22:51:35.467143 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260806 22:51:35.467614 5474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:35.458512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36934 (local address 127.6.40.124:39017)
0806 22:51:35.458549 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:35.458551 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:35.458567 (+ 16us) server_negotiation.cc:407] Connection header received
0806 22:51:35.458585 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:35.458585 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:35.458593 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:35.458644 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:35.458886 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:35.459527 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:35.460206 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:35.460501 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:35.460578 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:35.462467 (+ 1889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:51:35.462469 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:51:35.462469 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:51:35.462475 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:51:35.465985 (+ 3510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:35.466383 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:35.466384 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:35.466385 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:35.466447 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:35.466671 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:35.466672 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:35.466672 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:35.466813 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:51:35.466890 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:35.467017 (+ 127us) server_negotiation.cc:299] Negotiation successful
0806 22:51:35.467563 (+ 546us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":415,"server-negotiator.queue_time_us":22}
I20260806 22:51:35.468801 5374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:35.460094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57464 (local address 127.6.40.125:38863)
0806 22:51:35.460139 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:35.460141 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:35.460149 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:51:35.460172 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:35.460173 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:35.460183 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:35.460226 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:35.460823 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:35.461539 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:35.462259 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:35.462509 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:35.462577 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:35.465350 (+ 2773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:51:35.465351 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:51:35.465352 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:51:35.465360 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:51:35.467076 (+ 1716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:35.467552 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:35.467553 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:35.467554 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:35.467616 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:35.467951 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:35.467952 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:35.467953 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:35.468078 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:51:35.468246 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:35.468625 (+ 379us) server_negotiation.cc:299] Negotiation successful
0806 22:51:35.468708 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 22:51:35.469014 5259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:35.459374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.124:39017 (local address 127.0.0.1:36950)
0806 22:51:35.459564 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0806 22:51:35.459575 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:51:35.459615 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:51:35.460397 (+ 782us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:51:35.460398 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:51:35.460562 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:51:35.460608 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:35.460610 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:35.461627 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:35.461629 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:35.462448 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:35.462452 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:35.462942 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:35.462943 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:35.463030 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:35.463379 (+ 349us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:51:35.463383 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:51:35.465440 (+ 2057us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:51:35.467777 (+ 2337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:51:35.467778 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:51:35.467778 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:51:35.468017 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:51:35.468239 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:51:35.468240 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:51:35.468240 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:51:35.468306 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:51:35.468680 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:51:35.468683 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:51:35.468726 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 22:51:35.468800 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 22:51:35.468921 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176}
I20260806 22:51:35.469099 5476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:35.459422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36950 (local address 127.6.40.124:39017)
0806 22:51:35.460249 (+ 827us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:35.460250 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:35.460256 (+ 6us) server_negotiation.cc:407] Connection header received
0806 22:51:35.460270 (+ 14us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:35.460270 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:35.460279 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:35.460338 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:35.460705 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:35.461534 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:35.462533 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:35.462877 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:35.462941 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:35.465531 (+ 2590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:51:35.465532 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:51:35.465532 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:51:35.465539 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:51:35.467711 (+ 2172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:35.468091 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:35.468092 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:35.468093 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:35.468151 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:35.468390 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:35.468391 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:35.468391 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:35.468524 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:51:35.468609 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:35.468828 (+ 219us) server_negotiation.cc:299] Negotiation successful
0806 22:51:35.468969 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":807,"thread_start_us":120,"threads_started":1}
I20260806 22:51:35.469388 5469 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de4cafdc09a84f2e98fc738b9df0fce5: Bootstrap starting.
I20260806 22:51:35.470095 5469 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P de4cafdc09a84f2e98fc738b9df0fce5: Neither blocks nor log segments found. Creating new log.
I20260806 22:51:35.470319 5469 log.cc:824] T 00000000000000000000000000000000 P de4cafdc09a84f2e98fc738b9df0fce5: Log is configured to *not* fsync() on all Append() calls
I20260806 22:51:35.470646 5256 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9be425d006c470d891f1f95f373b003: Bootstrap starting.
I20260806 22:51:35.471110 5469 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de4cafdc09a84f2e98fc738b9df0fce5: No bootstrap required, opened a new log
I20260806 22:51:35.471302 5256 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a9be425d006c470d891f1f95f373b003: Neither blocks nor log segments found. Creating new log.
I20260806 22:51:35.471597 5256 log.cc:824] T 00000000000000000000000000000000 P a9be425d006c470d891f1f95f373b003: Log is configured to *not* fsync() on all Append() calls
I20260806 22:51:35.472430 5256 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9be425d006c470d891f1f95f373b003: No bootstrap required, opened a new log
I20260806 22:51:35.472513 6305 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:51:35.473131 5469 raft_consensus.cc:348] T 00000000000000000000000000000000 P de4cafdc09a84f2e98fc738b9df0fce5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9be425d006c470d891f1f95f373b003" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 38461 } } peers { permanent_uuid: "78ab2695431f473198a5fcbbf4a71642" member_type: VOTER last_known_addr { host: "127.6.40.125" port: 38863 } } peers { permanent_uuid: "de4cafdc09a84f2e98fc738b9df0fce5" member_type: VOTER last_known_addr { host: "127.6.40.124" port: 39017 } }
I20260806 22:51:35.473299 5469 raft_consensus.cc:729] T 00000000000000000000000000000000 P de4cafdc09a84f2e98fc738b9df0fce5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de4cafdc09a84f2e98fc738b9df0fce5, State: Initialized, Role: FOLLOWER
I20260806 22:51:35.473387 5469 consensus_queue.cc:260] T 00000000000000000000000000000000 P de4cafdc09a84f2e98fc738b9df0fce5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9be425d006c470d891f1f95f373b003" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 38461 } } peers { permanent_uuid: "78ab2695431f473198a5fcbbf4a71642" member_type: VOTER last_known_addr { host: "127.6.40.125" port: 38863 } } peers { permanent_uuid: "de4cafdc09a84f2e98fc738b9df0fce5" member_type: VOTER last_known_addr { host: "127.6.40.124" port: 39017 } }
I20260806 22:51:35.473590 5223 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "a9be425d006c470d891f1f95f373b003"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59466
I20260806 22:51:35.473708 5469 sys_catalog.cc:564] T 00000000000000000000000000000000 P de4cafdc09a84f2e98fc738b9df0fce5 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:51:35.473688 5481 sys_catalog.cc:455] T 00000000000000000000000000000000 P de4cafdc09a84f2e98fc738b9df0fce5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9be425d006c470d891f1f95f373b003" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 38461 } } peers { permanent_uuid: "78ab2695431f473198a5fcbbf4a71642" member_type: VOTER last_known_addr { host: "127.6.40.125" port: 38863 } } peers { permanent_uuid: "de4cafdc09a84f2e98fc738b9df0fce5" member_type: VOTER last_known_addr { host: "127.6.40.124" port: 39017 } } }
I20260806 22:51:35.473794 5481 sys_catalog.cc:458] T 00000000000000000000000000000000 P de4cafdc09a84f2e98fc738b9df0fce5 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:51:35.474870 5256 raft_consensus.cc:348] T 00000000000000000000000000000000 P a9be425d006c470d891f1f95f373b003 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9be425d006c470d891f1f95f373b003" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 38461 } } peers { permanent_uuid: "78ab2695431f473198a5fcbbf4a71642" member_type: VOTER last_known_addr { host: "127.6.40.125" port: 38863 } } peers { permanent_uuid: "de4cafdc09a84f2e98fc738b9df0fce5" member_type: VOTER last_known_addr { host: "127.6.40.124" port: 39017 } }
I20260806 22:51:35.475031 5256 raft_consensus.cc:729] T 00000000000000000000000000000000 P a9be425d006c470d891f1f95f373b003 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9be425d006c470d891f1f95f373b003, State: Initialized, Role: FOLLOWER
I20260806 22:51:35.475136 5256 consensus_queue.cc:260] T 00000000000000000000000000000000 P a9be425d006c470d891f1f95f373b003 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9be425d006c470d891f1f95f373b003" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 38461 } } peers { permanent_uuid: "78ab2695431f473198a5fcbbf4a71642" member_type: VOTER last_known_addr { host: "127.6.40.125" port: 38863 } } peers { permanent_uuid: "de4cafdc09a84f2e98fc738b9df0fce5" member_type: VOTER last_known_addr { host: "127.6.40.124" port: 39017 } }
I20260806 22:51:35.475600 5256 sys_catalog.cc:564] T 00000000000000000000000000000000 P a9be425d006c470d891f1f95f373b003 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:51:35.475601 5499 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9be425d006c470d891f1f95f373b003 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9be425d006c470d891f1f95f373b003" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 38461 } } peers { permanent_uuid: "78ab2695431f473198a5fcbbf4a71642" member_type: VOTER last_known_addr { host: "127.6.40.125" port: 38863 } } peers { permanent_uuid: "de4cafdc09a84f2e98fc738b9df0fce5" member_type: VOTER last_known_addr { host: "127.6.40.124" port: 39017 } } }
I20260806 22:51:35.475710 5223 raft_consensus.cc:482] T 00000000000000000000000000000000 P a9be425d006c470d891f1f95f373b003 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260806 22:51:35.475720 5499 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9be425d006c470d891f1f95f373b003 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:51:35.475780 5223 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a9be425d006c470d891f1f95f373b003 [term 0 FOLLOWER]: Advancing to term 1
W20260806 22:51:35.476045 5497 catalog_manager.cc:1558] T 00000000000000000000000000000000 P de4cafdc09a84f2e98fc738b9df0fce5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:51:35.476141 5497 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:51:35.476714 5223 raft_consensus.cc:504] T 00000000000000000000000000000000 P a9be425d006c470d891f1f95f373b003 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9be425d006c470d891f1f95f373b003" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 38461 } } peers { permanent_uuid: "78ab2695431f473198a5fcbbf4a71642" member_type: VOTER last_known_addr { host: "127.6.40.125" port: 38863 } } peers { permanent_uuid: "de4cafdc09a84f2e98fc738b9df0fce5" member_type: VOTER last_known_addr { host: "127.6.40.124" port: 39017 } }
I20260806 22:51:35.477015 5223 leader_election.cc:288] T 00000000000000000000000000000000 P a9be425d006c470d891f1f95f373b003 [CANDIDATE]: Term 1 election: Requested vote from peers 78ab2695431f473198a5fcbbf4a71642 (127.6.40.125:38863), de4cafdc09a84f2e98fc738b9df0fce5 (127.6.40.124:39017)
I20260806 22:51:35.478266 5431 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a9be425d006c470d891f1f95f373b003" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "de4cafdc09a84f2e98fc738b9df0fce5"
I20260806 22:51:35.478421 5431 raft_consensus.cc:3037] T 00000000000000000000000000000000 P de4cafdc09a84f2e98fc738b9df0fce5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:51:35.479348 5431 raft_consensus.cc:2454] T 00000000000000000000000000000000 P de4cafdc09a84f2e98fc738b9df0fce5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9be425d006c470d891f1f95f373b003 in term 1.
I20260806 22:51:35.479727 5193 leader_election.cc:302] T 00000000000000000000000000000000 P a9be425d006c470d891f1f95f373b003 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9be425d006c470d891f1f95f373b003, de4cafdc09a84f2e98fc738b9df0fce5; no voters:
I20260806 22:51:35.476017 5377 master_runner.cc:186] Error getting master registration for 127.6.40.126:38461: OK, instance_id { permanent_uuid: "a9be425d006c470d891f1f95f373b003" instance_seqno: 1786056695113933 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 22:51:35.480244 5377 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 22:51:35.480353 5499 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a9be425d006c470d891f1f95f373b003 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:51:35.480521 5499 raft_consensus.cc:686] T 00000000000000000000000000000000 P a9be425d006c470d891f1f95f373b003 [term 1 LEADER]: Becoming Leader. State: Replica: a9be425d006c470d891f1f95f373b003, State: Running, Role: LEADER
I20260806 22:51:35.480615 5499 consensus_queue.cc:237] T 00000000000000000000000000000000 P a9be425d006c470d891f1f95f373b003 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9be425d006c470d891f1f95f373b003" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 38461 } } peers { permanent_uuid: "78ab2695431f473198a5fcbbf4a71642" member_type: VOTER last_known_addr { host: "127.6.40.125" port: 38863 } } peers { permanent_uuid: "de4cafdc09a84f2e98fc738b9df0fce5" member_type: VOTER last_known_addr { host: "127.6.40.124" port: 39017 } }
I20260806 22:51:35.481127 5520 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9be425d006c470d891f1f95f373b003 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a9be425d006c470d891f1f95f373b003. Latest consensus state: current_term: 1 leader_uuid: "a9be425d006c470d891f1f95f373b003" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9be425d006c470d891f1f95f373b003" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 38461 } } peers { permanent_uuid: "78ab2695431f473198a5fcbbf4a71642" member_type: VOTER last_known_addr { host: "127.6.40.125" port: 38863 } } peers { permanent_uuid: "de4cafdc09a84f2e98fc738b9df0fce5" member_type: VOTER last_known_addr { host: "127.6.40.124" port: 39017 } } }
I20260806 22:51:35.481248 5520 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9be425d006c470d891f1f95f373b003 [sys.catalog]: This master's current role is: LEADER
I20260806 22:51:35.478633 5166 master_runner.cc:186] Error getting master registration for 127.6.40.125:38863: OK, instance_id { permanent_uuid: "78ab2695431f473198a5fcbbf4a71642" instance_seqno: 1786056695271944 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 22:51:35.481650 5166 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 22:51:35.482496 5523 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:51:35.483026 5523 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:51:35.487694 5261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:35.478356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59476 (local address 127.6.40.126:38461)
0806 22:51:35.478394 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:35.478396 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:35.478662 (+ 266us) server_negotiation.cc:407] Connection header received
0806 22:51:35.478754 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:35.478755 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:35.478767 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:35.478796 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:35.479596 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:35.480276 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:35.482415 (+ 2139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:35.482698 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:35.482782 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:35.484767 (+ 1985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:51:35.484769 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:51:35.484769 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:51:35.484777 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:51:35.486451 (+ 1674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:35.486852 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:35.486854 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:35.486854 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:35.486916 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:35.487112 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:35.487113 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:35.487113 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:35.487256 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:51:35.487372 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:35.487513 (+ 141us) server_negotiation.cc:299] Negotiation successful
0806 22:51:35.487597 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 22:51:35.496474 5374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:35.483330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57470 (local address 127.6.40.125:38863)
0806 22:51:35.483382 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:35.483384 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:35.487729 (+ 4345us) server_negotiation.cc:407] Connection header received
0806 22:51:35.487841 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:35.487842 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:35.487855 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:35.487890 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:35.488558 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:35.489214 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:35.490178 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:35.490869 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:35.490959 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:35.492352 (+ 1393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:51:35.492354 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:51:35.492354 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:51:35.492361 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:51:35.494642 (+ 2281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:35.495291 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:35.495293 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:35.495293 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:35.495352 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:35.495681 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:35.495683 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:35.495683 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:35.495828 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:51:35.495901 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:35.496280 (+ 379us) server_negotiation.cc:299] Negotiation successful
0806 22:51:35.496371 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 22:51:35.497009 5476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:35.483708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36956 (local address 127.6.40.124:39017)
0806 22:51:35.483755 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:35.483757 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:35.489067 (+ 5310us) server_negotiation.cc:407] Connection header received
0806 22:51:35.489197 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:35.489199 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:35.489210 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:35.489241 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:35.490008 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:35.490749 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:35.491752 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:35.492084 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:35.492126 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:35.493849 (+ 1723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:51:35.493851 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:51:35.493851 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:51:35.493858 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:51:35.495676 (+ 1818us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:35.496046 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:35.496047 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:35.496047 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:35.496108 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:35.496316 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:35.496317 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:35.496317 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:35.496469 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:51:35.496572 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:35.496795 (+ 223us) server_negotiation.cc:299] Negotiation successful
0806 22:51:35.496922 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 22:51:35.498355 5431 raft_consensus.cc:1261] T 00000000000000000000000000000000 P de4cafdc09a84f2e98fc738b9df0fce5 [term 1 FOLLOWER]: Refusing update from remote peer a9be425d006c470d891f1f95f373b003: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:51:35.498706 5520 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a9be425d006c470d891f1f95f373b003 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4cafdc09a84f2e98fc738b9df0fce5" member_type: VOTER last_known_addr { host: "127.6.40.124" port: 39017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:51:35.500480 5481 sys_catalog.cc:455] T 00000000000000000000000000000000 P de4cafdc09a84f2e98fc738b9df0fce5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a9be425d006c470d891f1f95f373b003. Latest consensus state: current_term: 1 leader_uuid: "a9be425d006c470d891f1f95f373b003" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9be425d006c470d891f1f95f373b003" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 38461 } } peers { permanent_uuid: "78ab2695431f473198a5fcbbf4a71642" member_type: VOTER last_known_addr { host: "127.6.40.125" port: 38863 } } peers { permanent_uuid: "de4cafdc09a84f2e98fc738b9df0fce5" member_type: VOTER last_known_addr { host: "127.6.40.124" port: 39017 } } }
I20260806 22:51:35.500578 5481 sys_catalog.cc:458] T 00000000000000000000000000000000 P de4cafdc09a84f2e98fc738b9df0fce5 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:51:35.506165 5499 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9be425d006c470d891f1f95f373b003 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a9be425d006c470d891f1f95f373b003" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9be425d006c470d891f1f95f373b003" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 38461 } } peers { permanent_uuid: "78ab2695431f473198a5fcbbf4a71642" member_type: VOTER last_known_addr { host: "127.6.40.125" port: 38863 } } peers { permanent_uuid: "de4cafdc09a84f2e98fc738b9df0fce5" member_type: VOTER last_known_addr { host: "127.6.40.124" port: 39017 } } }
I20260806 22:51:35.506278 5499 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9be425d006c470d891f1f95f373b003 [sys.catalog]: This master's current role is: LEADER
I20260806 22:51:35.506297 5481 sys_catalog.cc:455] T 00000000000000000000000000000000 P de4cafdc09a84f2e98fc738b9df0fce5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a9be425d006c470d891f1f95f373b003" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9be425d006c470d891f1f95f373b003" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 38461 } } peers { permanent_uuid: "78ab2695431f473198a5fcbbf4a71642" member_type: VOTER last_known_addr { host: "127.6.40.125" port: 38863 } } peers { permanent_uuid: "de4cafdc09a84f2e98fc738b9df0fce5" member_type: VOTER last_known_addr { host: "127.6.40.124" port: 39017 } } }
I20260806 22:51:35.506371 5481 sys_catalog.cc:458] T 00000000000000000000000000000000 P de4cafdc09a84f2e98fc738b9df0fce5 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:51:35.506417 5528 mvcc.cc:205] Tried to move back new op lower bound from 7315688224758571008 to 7315688224689184768. Current Snapshot: MvccSnapshot[applied={T|T < 7315688224758571008}]
I20260806 22:51:35.506834 5529 mvcc.cc:205] Tried to move back new op lower bound from 7315688224758571008 to 7315688224689184768. Current Snapshot: MvccSnapshot[applied={T|T < 7315688224758571008}]
I20260806 22:51:35.507467 5523 catalog_manager.cc:1347] Generated new cluster ID: 069e9e66b9cf4b9bb1e4dd9e5d518aed
I20260806 22:51:35.507524 5523 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:51:35.508885 5370 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.40.124" port: 39017 } attempt: 3
Aug 06 22:51:35 dist-test-slave-zh2d krb5kdc[5150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056695, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.125@KRBTEST.COM for kudu/127.6.40.124@KRBTEST.COM
I20260806 22:51:35.517359 5373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:35.509163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.124:39017 (local address 127.0.0.1:36958)
0806 22:51:35.509249 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0806 22:51:35.509259 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:51:35.509293 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:51:35.509433 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:51:35.509434 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:51:35.509619 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:51:35.509669 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:35.509671 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:35.510624 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:35.510625 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:35.511426 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:35.511428 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:35.512047 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:35.512049 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:35.512124 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:35.512674 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:51:35.512679 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:51:35.514271 (+ 1592us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:51:35.516224 (+ 1953us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:51:35.516225 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:51:35.516225 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:51:35.516463 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:51:35.516653 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:51:35.516654 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:51:35.516655 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:51:35.516721 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:51:35.517010 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:51:35.517011 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:51:35.517060 (+ 49us) client_negotiation.cc:769] Sending connection context
0806 22:51:35.517133 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 22:51:35.517248 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69}
I20260806 22:51:35.517724 5476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:35.509211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36958 (local address 127.6.40.124:39017)
0806 22:51:35.509278 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:35.509280 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:35.509290 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:51:35.509342 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:35.509342 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:35.509352 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:35.509432 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:35.509733 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:35.510534 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:35.511596 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:35.511913 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:35.511965 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:35.514382 (+ 2417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:51:35.514384 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:51:35.514384 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:51:35.514391 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:51:35.516152 (+ 1761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:35.516539 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:35.516540 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:35.516541 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:35.516606 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:35.516772 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:35.516773 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:35.516773 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:35.516958 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:51:35.517015 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:35.517556 (+ 541us) server_negotiation.cc:299] Negotiation successful
0806 22:51:35.517637 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 22:51:35.519256 5370 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78ab2695431f473198a5fcbbf4a71642: Bootstrap starting.
I20260806 22:51:35.520002 5370 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 78ab2695431f473198a5fcbbf4a71642: Neither blocks nor log segments found. Creating new log.
I20260806 22:51:35.520380 5370 log.cc:824] T 00000000000000000000000000000000 P 78ab2695431f473198a5fcbbf4a71642: Log is configured to *not* fsync() on all Append() calls
I20260806 22:51:35.522117 5370 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78ab2695431f473198a5fcbbf4a71642: No bootstrap required, opened a new log
I20260806 22:51:35.524258 5370 raft_consensus.cc:348] T 00000000000000000000000000000000 P 78ab2695431f473198a5fcbbf4a71642 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9be425d006c470d891f1f95f373b003" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 38461 } } peers { permanent_uuid: "78ab2695431f473198a5fcbbf4a71642" member_type: VOTER last_known_addr { host: "127.6.40.125" port: 38863 } } peers { permanent_uuid: "de4cafdc09a84f2e98fc738b9df0fce5" member_type: VOTER last_known_addr { host: "127.6.40.124" port: 39017 } }
I20260806 22:51:35.524400 5370 raft_consensus.cc:729] T 00000000000000000000000000000000 P 78ab2695431f473198a5fcbbf4a71642 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78ab2695431f473198a5fcbbf4a71642, State: Initialized, Role: FOLLOWER
I20260806 22:51:35.524497 5370 consensus_queue.cc:260] T 00000000000000000000000000000000 P 78ab2695431f473198a5fcbbf4a71642 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9be425d006c470d891f1f95f373b003" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 38461 } } peers { permanent_uuid: "78ab2695431f473198a5fcbbf4a71642" member_type: VOTER last_known_addr { host: "127.6.40.125" port: 38863 } } peers { permanent_uuid: "de4cafdc09a84f2e98fc738b9df0fce5" member_type: VOTER last_known_addr { host: "127.6.40.124" port: 39017 } }
I20260806 22:51:35.524806 5535 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78ab2695431f473198a5fcbbf4a71642 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9be425d006c470d891f1f95f373b003" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 38461 } } peers { permanent_uuid: "78ab2695431f473198a5fcbbf4a71642" member_type: VOTER last_known_addr { host: "127.6.40.125" port: 38863 } } peers { permanent_uuid: "de4cafdc09a84f2e98fc738b9df0fce5" member_type: VOTER last_known_addr { host: "127.6.40.124" port: 39017 } } }
I20260806 22:51:35.524941 5328 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 78ab2695431f473198a5fcbbf4a71642 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:51:35.525074 5535 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78ab2695431f473198a5fcbbf4a71642 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:51:35.525775 5328 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 78ab2695431f473198a5fcbbf4a71642 [term 1 FOLLOWER]: Refusing update from remote peer a9be425d006c470d891f1f95f373b003: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:51:35.526095 5520 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a9be425d006c470d891f1f95f373b003 [LEADER]: Connected to new peer: Peer: permanent_uuid: "78ab2695431f473198a5fcbbf4a71642" member_type: VOTER last_known_addr { host: "127.6.40.125" port: 38863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:51:35.527822 5535 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78ab2695431f473198a5fcbbf4a71642 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a9be425d006c470d891f1f95f373b003. Latest consensus state: current_term: 1 leader_uuid: "a9be425d006c470d891f1f95f373b003" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9be425d006c470d891f1f95f373b003" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 38461 } } peers { permanent_uuid: "78ab2695431f473198a5fcbbf4a71642" member_type: VOTER last_known_addr { host: "127.6.40.125" port: 38863 } } peers { permanent_uuid: "de4cafdc09a84f2e98fc738b9df0fce5" member_type: VOTER last_known_addr { host: "127.6.40.124" port: 39017 } } }
I20260806 22:51:35.527910 5535 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78ab2695431f473198a5fcbbf4a71642 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:51:35.524878 5370 sys_catalog.cc:564] T 00000000000000000000000000000000 P 78ab2695431f473198a5fcbbf4a71642 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:51:35.524905 5330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a9be425d006c470d891f1f95f373b003" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "78ab2695431f473198a5fcbbf4a71642"
I20260806 22:51:35.529068 5523 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:51:35.529812 5523 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:51:35.529978 5540 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78ab2695431f473198a5fcbbf4a71642 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a9be425d006c470d891f1f95f373b003" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9be425d006c470d891f1f95f373b003" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 38461 } } peers { permanent_uuid: "78ab2695431f473198a5fcbbf4a71642" member_type: VOTER last_known_addr { host: "127.6.40.125" port: 38863 } } peers { permanent_uuid: "de4cafdc09a84f2e98fc738b9df0fce5" member_type: VOTER last_known_addr { host: "127.6.40.124" port: 39017 } } }
I20260806 22:51:35.530068 5540 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78ab2695431f473198a5fcbbf4a71642 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:51:35.533533 5560 catalog_manager.cc:1259] Loaded cluster ID: 069e9e66b9cf4b9bb1e4dd9e5d518aed
I20260806 22:51:35.533586 5560 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 78ab2695431f473198a5fcbbf4a71642: loading cluster ID for follower catalog manager: success
I20260806 22:51:35.534637 5560 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 78ab2695431f473198a5fcbbf4a71642: acquiring CA information for follower catalog manager: success
W20260806 22:51:35.534752 5560 catalog_manager.cc:873] Not found: no TSK found in the system table: failed to prepare follower catalog manager, will retry
I20260806 22:51:35.536309 5330 raft_consensus.cc:2396] T 00000000000000000000000000000000 P 78ab2695431f473198a5fcbbf4a71642 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate a9be425d006c470d891f1f95f373b003 for term 1 because replica has last-logged OpId of term: 1 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260806 22:51:35.536911 5520 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9be425d006c470d891f1f95f373b003 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a9be425d006c470d891f1f95f373b003" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9be425d006c470d891f1f95f373b003" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 38461 } } peers { permanent_uuid: "78ab2695431f473198a5fcbbf4a71642" member_type: VOTER last_known_addr { host: "127.6.40.125" port: 38863 } } peers { permanent_uuid: "de4cafdc09a84f2e98fc738b9df0fce5" member_type: VOTER last_known_addr { host: "127.6.40.124" port: 39017 } } }
I20260806 22:51:35.537009 5520 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9be425d006c470d891f1f95f373b003 [sys.catalog]: This master's current role is: LEADER
I20260806 22:51:35.537422 5523 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a9be425d006c470d891f1f95f373b003: Generated new TSK 0
I20260806 22:51:35.537608 5523 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 22:51:35.562026 5374 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:51:35.560346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57478 (local address 127.6.40.125:38863)
0806 22:51:35.560402 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:35.560405 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:35.561338 (+ 933us) server_negotiation.cc:407] Connection header received
0806 22:51:35.561460 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:35.561461 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:35.561473 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:35.561504 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:35.561956 (+ 452us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57478: BlockingRecv error: recv got EOF from 127.0.0.1:57478 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
W20260806 22:51:35.563167 5476 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:51:35.560706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36974 (local address 127.6.40.124:39017)
0806 22:51:35.560754 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:35.560756 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:35.562254 (+ 1498us) server_negotiation.cc:407] Connection header received
0806 22:51:35.562498 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:35.562499 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:35.562511 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:35.562542 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:35.563100 (+ 558us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36974: BlockingRecv error: recv got EOF from 127.0.0.1:36974 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260806 22:51:35.563796 5261 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:51:35.561072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59480 (local address 127.6.40.126:38461)
0806 22:51:35.561114 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:35.561116 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:35.562445 (+ 1329us) server_negotiation.cc:407] Connection header received
0806 22:51:35.563388 (+ 943us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:35.563389 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:35.563401 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:35.563431 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:35.563732 (+ 301us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59480: BlockingRecv error: recv got EOF from 127.0.0.1:59480 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260806 22:51:35.569918 5374 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:51:35.569204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57490 (local address 127.6.40.125:38863)
0806 22:51:35.569242 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:35.569244 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:35.569350 (+ 106us) server_negotiation.cc:407] Connection header received
0806 22:51:35.569475 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:35.569475 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:35.569496 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:35.569525 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:35.569857 (+ 332us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57490: BlockingRecv error: recv got EOF from 127.0.0.1:57490 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260806 22:51:35.576112 5374 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:51:35.575177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57494 (local address 127.6.40.125:38863)
0806 22:51:35.575212 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:35.575214 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:35.575539 (+ 325us) server_negotiation.cc:407] Connection header received
0806 22:51:35.575636 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:35.575636 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:35.575647 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:35.575676 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:35.576060 (+ 384us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57494: BlockingRecv error: recv got EOF from 127.0.0.1:57494 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260806 22:51:35.579968 5374 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:51:35.579170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57510 (local address 127.6.40.125:38863)
0806 22:51:35.579205 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:35.579206 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:35.579489 (+ 283us) server_negotiation.cc:407] Connection header received
0806 22:51:35.579578 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:35.579579 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:35.579589 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:35.579616 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:35.579920 (+ 304us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57510: BlockingRecv error: recv got EOF from 127.0.0.1:57510 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260806 22:51:35.585831 5374 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:51:35.585011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57526 (local address 127.6.40.125:38863)
0806 22:51:35.585046 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:35.585048 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:35.585294 (+ 246us) server_negotiation.cc:407] Connection header received
0806 22:51:35.585407 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:35.585408 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:35.585418 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:35.585450 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:35.585783 (+ 333us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57526: BlockingRecv error: recv got EOF from 127.0.0.1:57526 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260806 22:51:35.596400 5374 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:51:35.595376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57530 (local address 127.6.40.125:38863)
0806 22:51:35.595449 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:35.595451 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:35.595794 (+ 343us) server_negotiation.cc:407] Connection header received
0806 22:51:35.595903 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:35.595904 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:35.595916 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:35.595946 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:35.596317 (+ 371us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57530: BlockingRecv error: recv got EOF from 127.0.0.1:57530 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260806 22:51:35.614684 5374 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:51:35.613724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57544 (local address 127.6.40.125:38863)
0806 22:51:35.613767 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:35.613769 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:35.614072 (+ 303us) server_negotiation.cc:407] Connection header received
0806 22:51:35.614178 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:35.614179 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:35.614191 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:35.614223 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:35.614625 (+ 402us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57544: BlockingRecv error: recv got EOF from 127.0.0.1:57544 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260806 22:51:35.649083 5374 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:51:35.648039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57554 (local address 127.6.40.125:38863)
0806 22:51:35.648090 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:35.648092 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:35.648440 (+ 348us) server_negotiation.cc:407] Connection header received
0806 22:51:35.648548 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:35.648549 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:35.648561 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:35.648592 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:35.649017 (+ 425us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57554: BlockingRecv error: recv got EOF from 127.0.0.1:57554 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260806 22:51:35.715346 5374 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:51:35.714516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57562 (local address 127.6.40.125:38863)
0806 22:51:35.714574 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:35.714576 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:35.714673 (+ 97us) server_negotiation.cc:407] Connection header received
0806 22:51:35.714709 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:35.714710 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:35.714723 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:35.714812 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:35.715272 (+ 460us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57562: BlockingRecv error: recv got EOF from 127.0.0.1:57562 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
W20260806 22:51:35.845423 5374 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:51:35.844571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57564 (local address 127.6.40.125:38863)
0806 22:51:35.844671 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:35.844673 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:35.844772 (+ 99us) server_negotiation.cc:407] Connection header received
0806 22:51:35.844908 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:35.844909 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:35.844922 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:35.844997 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:35.845358 (+ 361us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57564: BlockingRecv error: recv got EOF from 127.0.0.1:57564 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260806 22:51:36.103329 5374 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:51:36.102697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57576 (local address 127.6.40.125:38863)
0806 22:51:36.102782 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:36.102783 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:36.102792 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:51:36.102818 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:36.102819 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:36.102828 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:36.102889 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:36.103257 (+ 368us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57576: BlockingRecv error: recv got EOF from 127.0.0.1:57576 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 22:51:36.476706 5497 catalog_manager.cc:1259] Loaded cluster ID: 069e9e66b9cf4b9bb1e4dd9e5d518aed
I20260806 22:51:36.476804 5497 catalog_manager.cc:1552] T 00000000000000000000000000000000 P de4cafdc09a84f2e98fc738b9df0fce5: loading cluster ID for follower catalog manager: success
I20260806 22:51:36.477772 5497 catalog_manager.cc:1574] T 00000000000000000000000000000000 P de4cafdc09a84f2e98fc738b9df0fce5: acquiring CA information for follower catalog manager: success
I20260806 22:51:36.478106 5497 catalog_manager.cc:1602] T 00000000000000000000000000000000 P de4cafdc09a84f2e98fc738b9df0fce5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 22:51:36.535341 5560 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 78ab2695431f473198a5fcbbf4a71642: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 22:51:36.619287 5646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:36.616625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57582 (local address 127.6.40.125:38863)
0806 22:51:36.616737 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:36.616738 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:36.616749 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:51:36.616778 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:36.616779 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:36.616831 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:36.616969 (+ 138us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:51:36.617227 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:36.617771 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:36.618549 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:36.618755 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:36.618821 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:36.618981 (+ 160us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:51:36.619097 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:51:36.619160 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:36.619198 (+ 38us) server_negotiation.cc:299] Negotiation successful
0806 22:51:36.619219 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":62,"threads_started":1}
I20260806 22:51:36.623620 5657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:36.621369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36984 (local address 127.6.40.124:39017)
0806 22:51:36.621476 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:36.621477 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:36.621487 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:51:36.621557 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:36.621557 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:36.621597 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:36.621664 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:51:36.621748 (+ 84us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:36.622234 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:36.622928 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:36.623125 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:36.623177 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:36.623293 (+ 116us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:51:36.623392 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:51:36.623442 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:36.623491 (+ 49us) server_negotiation.cc:299] Negotiation successful
0806 22:51:36.623506 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81,"thread_start_us":59,"threads_started":1}
I20260806 22:51:36.628188 5646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:36.625244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57592 (local address 127.6.40.125:38863)
0806 22:51:36.625288 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:36.625290 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:36.625440 (+ 150us) server_negotiation.cc:407] Connection header received
0806 22:51:36.625570 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:36.625571 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:36.625582 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:36.625626 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:51:36.625768 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:36.626521 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:36.627415 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:36.627731 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:36.627802 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:36.627928 (+ 126us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:51:36.627963 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:51:36.628015 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:36.628072 (+ 57us) server_negotiation.cc:299] Negotiation successful
0806 22:51:36.628095 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 22:51:36.629004 5666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:36.625289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59490 (local address 127.6.40.126:38461)
0806 22:51:36.625402 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:36.625404 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:36.625414 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:51:36.625519 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:36.625520 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:36.625577 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:36.625666 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:51:36.625860 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:36.626615 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:36.627713 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:36.628050 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:36.628108 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:36.628588 (+ 480us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:51:36.628688 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:51:36.628780 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:36.628875 (+ 95us) server_negotiation.cc:299] Negotiation successful
0806 22:51:36.628908 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89,"thread_start_us":50,"threads_started":1}
I20260806 22:51:36.629619 5657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:36.625481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36994 (local address 127.6.40.124:39017)
0806 22:51:36.625525 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:36.625526 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:36.625762 (+ 236us) server_negotiation.cc:407] Connection header received
0806 22:51:36.625836 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:36.625837 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:36.625848 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:36.625900 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:51:36.626023 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:36.626475 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:36.627584 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:36.627875 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:36.627962 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:36.628099 (+ 137us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:51:36.628134 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:51:36.628165 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:36.628279 (+ 114us) server_negotiation.cc:299] Negotiation successful
0806 22:51:36.629542 (+ 1263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 22:51:36.631173 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 5166
I20260806 22:51:36.647316 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 5266
I20260806 22:51:36.655998 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 5377
2026-08-06T22:51:36Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3002 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056405724609-6305-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:51:36 dist-test-slave-zh2d krb5kdc[5670](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:51:36 dist-test-slave-zh2d krb5kdc[5670](info): set up 2 sockets
Aug 06 22:51:36 dist-test-slave-zh2d krb5kdc[5670](info): commencing operation
krb5kdc: starting...
W20260806 22:51:37.949592 6305 mini_kdc.cc:121] Time spent starting KDC: real 1.270s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056405724609-6305-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056405724609-6305-0/krb5kdc/test-user.keytab.
Aug 06 22:51:38 dist-test-slave-zh2d krb5kdc[5670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056698, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:51:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:51:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.40.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.40.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.40.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab.
Entry for principal kudu/127.6.40.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab.
I20260806 22:51:38.084506 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056405724609-6305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056405724609-6305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056405724609-6305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.126:41601
--webserver_interface=127.6.40.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab
--principal=kudu/127.6.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:37945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.40.126:41601,127.6.40.125:46517,127.6.40.124:40913
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:51:38.168401 5687 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 22:51:38.168675 5687 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:51:38.168707 5687 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:51:38.168752 5687 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:51:38.170051 5687 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:51:38.170125 5687 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:51:38.170149 5687 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:51:38.170188 5687 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:51:38.170220 5687 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:51:38.171684 5687 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:37945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056405724609-6305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056405724609-6305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.40.126:41601,127.6.40.125:46517,127.6.40.124:40913
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.126.keytab
--principal=kudu/127.6.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.126:41601
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5687
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056405724609-6305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:51:38.171964 5687 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:51:38.172199 5687 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:51:38.177150 5702 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:51:38.177150 5695 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:51:38.177170 5696 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:51:38.177409 5687 server_base.cc:1034] running on GCE node
I20260806 22:51:38.177706 5687 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:51:38.178030 5687 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:51:38.179178 5687 hybrid_clock.cc:648] HybridClock initialized: now 1786056698179167 us; error 30 us; skew 500 ppm
Aug 06 22:51:38 dist-test-slave-zh2d krb5kdc[5670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056698, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:51:38.189064 5687 init.cc:377] Logged in from keytab as kudu/127.6.40.126@KRBTEST.COM (short username kudu)
I20260806 22:51:38.189522 5687 webserver.cc:460] Webserver started at http://127.6.40.126:35265/ using document root <none> and password file <none>
I20260806 22:51:38.189790 5687 fs_manager.cc:359] Metadata directory not provided
I20260806 22:51:38.189865 5687 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:51:38.190024 5687 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:51:38.191120 5687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056405724609-6305-0/minicluster-data/master-0/data/instance:
uuid: "dbc514d3a698458fbf0eb2f5e304d5ef"
format_stamp: "Formatted at 2026-08-06 22:51:38 on dist-test-slave-zh2d"
server_key: "511ea2ce5fcd9441a4a6fe02ac49909b"
server_key_iv: "94614d151c2fcb6f4f2f0e7eb2787c34"
server_key_version: "encryptionkey@0"
I20260806 22:51:38.191507 5687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056405724609-6305-0/minicluster-data/master-0/wal/instance:
uuid: "dbc514d3a698458fbf0eb2f5e304d5ef"
format_stamp: "Formatted at 2026-08-06 22:51:38 on dist-test-slave-zh2d"
server_key: "511ea2ce5fcd9441a4a6fe02ac49909b"
server_key_iv: "94614d151c2fcb6f4f2f0e7eb2787c34"
server_key_version: "encryptionkey@0"
I20260806 22:51:38.192970 5687 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:51:38.193807 5710 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:51:38.194015 5687 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:51:38.194123 5687 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056405724609-6305-0/minicluster-data/master-0/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056405724609-6305-0/minicluster-data/master-0/wal
uuid: "dbc514d3a698458fbf0eb2f5e304d5ef"
format_stamp: "Formatted at 2026-08-06 22:51:38 on dist-test-slave-zh2d"
server_key: "511ea2ce5fcd9441a4a6fe02ac49909b"
server_key_iv: "94614d151c2fcb6f4f2f0e7eb2787c34"
server_key_version: "encryptionkey@0"
I20260806 22:51:38.194207 5687 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056405724609-6305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056405724609-6305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056405724609-6305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:51:38.207371 5687 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:51:38.209802 5687 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:51:38.209976 5687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:51:38.215034 5687 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.126:41601
I20260806 22:51:38.215086 5781 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.126:41601 every 8 connection(s)
I20260806 22:51:38.215463 5687 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056405724609-6305-0/minicluster-data/master-0/data/info.pb
I20260806 22:51:38.216463 5782 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:51:38.218129 5782 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.40.126" port: 41601 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:51:38.220439 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 5687
I20260806 22:51:38.220557 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056405724609-6305-0/minicluster-data/master-0/wal/instance
I20260806 22:51:38.220788 6305 external_mini_cluster.cc:1428] Setting key 7b3488e475e7be6b8e8cd4288663bab1
Aug 06 22:51:38 dist-test-slave-zh2d krb5kdc[5670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056698, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.126@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.40.125@KRBTEST.COM; defaulting to no policy
I20260806 22:51:38.228749 5785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:38.218843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:41601 (local address 127.0.0.1:50834)
0806 22:51:38.219010 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 22:51:38.219073 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0806 22:51:38.219584 (+ 511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:51:38.219893 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:51:38.219895 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:51:38.220173 (+ 278us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:51:38.220421 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:38.220428 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:38.221637 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:38.221638 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:38.222794 (+ 1156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:38.222798 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:38.223311 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:38.223312 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:38.223375 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:38.223928 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:51:38.223940 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:51:38.225381 (+ 1441us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:51:38.227201 (+ 1820us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:51:38.227208 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:51:38.227210 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:51:38.227413 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:51:38.227685 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:51:38.227686 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:51:38.227686 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:51:38.227774 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:51:38.228149 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:51:38.228155 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:51:38.228219 (+ 64us) client_negotiation.cc:769] Sending connection context
0806 22:51:38.228329 (+ 110us) client_negotiation.cc:240] Negotiation successful
0806 22:51:38.228573 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"mutex_wait_us":101,"thread_start_us":76,"threads_started":1}
I20260806 22:51:38.228751 5786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:38.219003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50834 (local address 127.6.40.126:41601)
0806 22:51:38.219118 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:38.219123 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:38.219157 (+ 34us) server_negotiation.cc:407] Connection header received
0806 22:51:38.219754 (+ 597us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:38.219756 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:38.219781 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:38.219875 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:38.220570 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:38.221531 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:38.222887 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:38.223199 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:38.223326 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:38.225473 (+ 2147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:51:38.225480 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:51:38.225481 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:51:38.225489 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:51:38.227091 (+ 1602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:38.227518 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:38.227520 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:38.227521 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:38.227607 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:38.227847 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:38.227848 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:38.227849 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:38.228067 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:51:38.228159 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:38.228364 (+ 205us) server_negotiation.cc:299] Negotiation successful
0806 22:51:38.228505 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85,"thread_start_us":39,"threads_started":1}
I20260806 22:51:38.229751 5782 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.40.125" port: 46517 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:51:38.230225 5785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:38.230039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.125:46517 (local address 127.0.0.1:55176)
0806 22:51:38.230096 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0806 22:51:38.230197 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.125:46517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260806 22:51:38.230482 5718 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.40.125:46517: connect: Connection refused (error 111)
I20260806 22:51:38.230914 5785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:38.230823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.125:46517 (local address 127.0.0.1:55180)
0806 22:51:38.230869 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0806 22:51:38.230896 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.125:46517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260806 22:51:38.231099 5782 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.40.125:46517: Network error: Client connection negotiation failed: client connection to 127.6.40.125:46517: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.40.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.40.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.125.keytab.
Entry for principal kudu/127.6.40.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.125.keytab.
I20260806 22:51:38.249807 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056405724609-6305-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056405724609-6305-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056405724609-6305-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056405724609-6305-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.125:46517
--webserver_interface=127.6.40.125
--webserver_port=0
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.125.keytab
--principal=kudu/127.6.40.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:37945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.40.126:41601,127.6.40.125:46517,127.6.40.124:40913
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:51:38.280320 5782 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.40.125" port: 46517 } attempt: 1
I20260806 22:51:38.280901 5785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:38.280708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.125:46517 (local address 127.0.0.1:55194)
0806 22:51:38.280786 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0806 22:51:38.280859 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.125:46517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260806 22:51:38.281316 5785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:38.281208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.125:46517 (local address 127.0.0.1:55208)
0806 22:51:38.281259 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0806 22:51:38.281291 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.125:46517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260806 22:51:38.281499 5782 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.40.125:46517: Network error: Client connection negotiation failed: client connection to 127.6.40.125:46517: connect: Connection refused (error 111)
W20260806 22:51:38.335950 5792 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 22:51:38.336165 5792 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:51:38.336184 5792 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:51:38.336203 5792 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:51:38.337407 5792 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:51:38.337450 5792 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:51:38.337464 5792 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:51:38.337497 5792 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:51:38.337523 5792 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:51:38.339000 5792 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:37945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056405724609-6305-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056405724609-6305-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.6.40.126:41601,127.6.40.125:46517,127.6.40.124:40913
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.125.keytab
--principal=kudu/127.6.40.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.125:46517
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056405724609-6305-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.6.40.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5792
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056405724609-6305-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:51:38.339253 5792 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:51:38.339488 5792 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:51:38.344614 5800 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:51:38.344691 5792 server_base.cc:1034] running on GCE node
W20260806 22:51:38.344648 5799 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:51:38.344650 5803 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:51:38.345085 5792 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:51:38.345400 5792 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:51:38.346552 5792 hybrid_clock.cc:648] HybridClock initialized: now 1786056698346537 us; error 28 us; skew 500 ppm
I20260806 22:51:38.349643 5782 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.40.125" port: 46517 } attempt: 2
I20260806 22:51:38.350056 5785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:38.349936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.125:46517 (local address 127.0.0.1:55212)
0806 22:51:38.349993 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0806 22:51:38.350028 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.125:46517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260806 22:51:38.350447 5785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:38.350306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.125:46517 (local address 127.0.0.1:55220)
0806 22:51:38.350378 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0806 22:51:38.350412 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.125:46517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260806 22:51:38.350579 5782 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.40.125:46517: Network error: Client connection negotiation failed: client connection to 127.6.40.125:46517: connect: Connection refused (error 111)
Aug 06 22:51:38 dist-test-slave-zh2d krb5kdc[5670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056698, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:51:38.354569 5792 init.cc:377] Logged in from keytab as kudu/127.6.40.125@KRBTEST.COM (short username kudu)
I20260806 22:51:38.355104 5792 webserver.cc:460] Webserver started at http://127.6.40.125:40607/ using document root <none> and password file <none>
I20260806 22:51:38.355369 5792 fs_manager.cc:359] Metadata directory not provided
I20260806 22:51:38.355448 5792 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:51:38.355602 5792 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:51:38.356693 5792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056405724609-6305-0/minicluster-data/master-1/data/instance:
uuid: "704c8f7e575f4354ac9d0bb5a6e7e432"
format_stamp: "Formatted at 2026-08-06 22:51:38 on dist-test-slave-zh2d"
server_key: "fb991562ded85c19a0982e95011073bb"
server_key_iv: "c16e007219f56174d819a71674f5d801"
server_key_version: "encryptionkey@0"
I20260806 22:51:38.357092 5792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056405724609-6305-0/minicluster-data/master-1/wal/instance:
uuid: "704c8f7e575f4354ac9d0bb5a6e7e432"
format_stamp: "Formatted at 2026-08-06 22:51:38 on dist-test-slave-zh2d"
server_key: "fb991562ded85c19a0982e95011073bb"
server_key_iv: "c16e007219f56174d819a71674f5d801"
server_key_version: "encryptionkey@0"
I20260806 22:51:38.358480 5792 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:51:38.359261 5814 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:51:38.359476 5792 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:51:38.359614 5792 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056405724609-6305-0/minicluster-data/master-1/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056405724609-6305-0/minicluster-data/master-1/wal
uuid: "704c8f7e575f4354ac9d0bb5a6e7e432"
format_stamp: "Formatted at 2026-08-06 22:51:38 on dist-test-slave-zh2d"
server_key: "fb991562ded85c19a0982e95011073bb"
server_key_iv: "c16e007219f56174d819a71674f5d801"
server_key_version: "encryptionkey@0"
I20260806 22:51:38.359692 5792 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056405724609-6305-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056405724609-6305-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056405724609-6305-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:51:38.376022 5792 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:51:38.378232 5792 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:51:38.378403 5792 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:51:38.382908 5792 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.125:46517
I20260806 22:51:38.382939 5883 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.125:46517 every 8 connection(s)
I20260806 22:51:38.383342 5792 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056405724609-6305-0/minicluster-data/master-1/data/info.pb
I20260806 22:51:38.384366 5887 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:51:38.385335 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 5792
I20260806 22:51:38.385504 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056405724609-6305-0/minicluster-data/master-1/wal/instance
I20260806 22:51:38.385737 6305 external_mini_cluster.cc:1428] Setting key d1b33f48f4f276338ab204bf2b3a5991
I20260806 22:51:38.386662 5887 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.40.126" port: 41601 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 22:51:38 dist-test-slave-zh2d krb5kdc[5670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056698, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.125@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.40.124@KRBTEST.COM; defaulting to no policy
I20260806 22:51:38.396968 5786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:38.387561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50846 (local address 127.6.40.126:41601)
0806 22:51:38.387620 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:38.387623 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:38.387914 (+ 291us) server_negotiation.cc:407] Connection header received
0806 22:51:38.388452 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:38.388453 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:38.388468 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:38.388521 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:38.389207 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:38.389711 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:38.390834 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:38.391053 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:38.391143 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:38.393717 (+ 2574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:51:38.393718 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:51:38.393719 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:51:38.393726 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:51:38.395335 (+ 1609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:38.395807 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:38.395808 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:38.395808 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:38.395873 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:38.396129 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:38.396130 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:38.396130 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:38.396247 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:51:38.396307 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:38.396745 (+ 438us) server_negotiation.cc:299] Negotiation successful
0806 22:51:38.396830 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 22:51:38.397171 5892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:38.387550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:41601 (local address 127.0.0.1:50846)
0806 22:51:38.387763 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0806 22:51:38.387818 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 22:51:38.388334 (+ 516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:51:38.388589 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:51:38.388592 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:51:38.388914 (+ 322us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:51:38.389109 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:38.389114 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:38.389813 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:38.389814 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:38.390744 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:38.390747 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:38.391179 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:38.391181 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:38.391294 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:38.391824 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:51:38.391833 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:51:38.393619 (+ 1786us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:51:38.395473 (+ 1854us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:51:38.395480 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:51:38.395482 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:51:38.395705 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:51:38.395956 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:51:38.395957 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:51:38.395958 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:51:38.396059 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:51:38.396337 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:51:38.396344 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:51:38.396538 (+ 194us) client_negotiation.cc:769] Sending connection context
0806 22:51:38.396687 (+ 149us) client_negotiation.cc:240] Negotiation successful
0806 22:51:38.396829 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":105,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.40.124@KRBTEST.COM" created.
I20260806 22:51:38.397800 5887 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.40.125" port: 46517 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 22:51:38 dist-test-slave-zh2d krb5kdc[5670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056698, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.125@KRBTEST.COM for kudu/127.6.40.125@KRBTEST.COM
I20260806 22:51:38.406100 5892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:38.398099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.125:46517 (local address 127.0.0.1:55230)
0806 22:51:38.398317 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0806 22:51:38.398329 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:51:38.398379 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:51:38.398546 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:51:38.398547 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:51:38.398693 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:51:38.398740 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:38.398741 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:38.399983 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:38.399984 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:38.400721 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:38.400724 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:38.401266 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:38.401267 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:38.401351 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:38.401741 (+ 390us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:51:38.401746 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:51:38.403145 (+ 1399us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:51:38.405008 (+ 1863us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:51:38.405009 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:51:38.405009 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:51:38.405179 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:51:38.405415 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:51:38.405415 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:51:38.405415 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:51:38.405472 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:51:38.405766 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:51:38.405766 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:51:38.405805 (+ 39us) client_negotiation.cc:769] Sending connection context
0806 22:51:38.405879 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 22:51:38.405982 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203}
I20260806 22:51:38.406169 5893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:38.398151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55230 (local address 127.6.40.125:46517)
0806 22:51:38.398332 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:38.398337 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:38.398420 (+ 83us) server_negotiation.cc:407] Connection header received
0806 22:51:38.398443 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:38.398445 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:38.398458 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:38.398546 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:38.398816 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:38.399882 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:38.400809 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:38.401183 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:38.401242 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:38.403244 (+ 2002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:51:38.403253 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:51:38.403254 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:51:38.403260 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:51:38.404915 (+ 1655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:38.405273 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:38.405277 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:38.405277 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:38.405335 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:38.405542 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:38.405542 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:38.405543 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:38.405685 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:51:38.405775 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:38.405913 (+ 138us) server_negotiation.cc:299] Negotiation successful
0806 22:51:38.406038 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":56,"server-negotiator.queue_time_us":143,"thread_start_us":115,"threads_started":1}
I20260806 22:51:38.406840 5887 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.40.124" port: 40913 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:51:38.407246 5892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:38.407074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.124:40913 (local address 127.0.0.1:36574)
0806 22:51:38.407140 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0806 22:51:38.407222 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.124:40913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
W20260806 22:51:38.407413 5815 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.40.124:40913: connect: Connection refused (error 111)
I20260806 22:51:38.407913 5892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:38.407818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.124:40913 (local address 127.0.0.1:36586)
0806 22:51:38.407864 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0806 22:51:38.407891 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.124:40913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260806 22:51:38.408046 5887 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.40.124:40913: Network error: Client connection negotiation failed: client connection to 127.6.40.124:40913: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.40.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.124.keytab.
Entry for principal kudu/127.6.40.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.124.keytab.
I20260806 22:51:38.413972 6305 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_hhfSb/build/release/bin/kudu
/tmp/dist-test-task_hhfSb/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056405724609-6305-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056405724609-6305-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056405724609-6305-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056405724609-6305-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.124:40913
--webserver_interface=127.6.40.124
--webserver_port=0
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.124.keytab
--principal=kudu/127.6.40.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.40.84:37945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.40.126:41601,127.6.40.125:46517,127.6.40.124:40913
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056405724609-6305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056405724609-6305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056405724609-6305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:51:38.441735 5782 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.40.125" port: 46517 } attempt: 3
Aug 06 22:51:38 dist-test-slave-zh2d krb5kdc[5670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056698, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.126@KRBTEST.COM for kudu/127.6.40.125@KRBTEST.COM
I20260806 22:51:38.449695 5785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:38.442157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.125:46517 (local address 127.0.0.1:55232)
0806 22:51:38.442223 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0806 22:51:38.442237 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 22:51:38.442317 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:51:38.442537 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:51:38.442538 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:51:38.442718 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:51:38.442789 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:38.442791 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:38.443488 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:38.443490 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:38.444047 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:38.444050 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:38.444468 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:38.444469 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:38.444567 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:38.445131 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:51:38.445136 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:51:38.446833 (+ 1697us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:51:38.448608 (+ 1775us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:51:38.448609 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:51:38.448609 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:51:38.448787 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:51:38.449014 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:51:38.449015 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:51:38.449015 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:51:38.449086 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:51:38.449351 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:51:38.449352 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:51:38.449403 (+ 51us) client_negotiation.cc:769] Sending connection context
0806 22:51:38.449483 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 22:51:38.449591 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260806 22:51:38.449694 5893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:38.442238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55232 (local address 127.6.40.125:46517)
0806 22:51:38.442312 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:38.442314 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:38.442327 (+ 13us) server_negotiation.cc:407] Connection header received
0806 22:51:38.442441 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:38.442442 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:38.442463 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:38.442531 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:38.442895 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:38.443400 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:38.444144 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:38.444389 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:38.444468 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:38.446957 (+ 2489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:51:38.446959 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:51:38.446959 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:51:38.446967 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:51:38.448522 (+ 1555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:38.448889 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:38.448890 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:38.448890 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:38.448947 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:38.449158 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:38.449160 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:38.449160 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:38.449276 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:51:38.449347 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:38.449522 (+ 175us) server_negotiation.cc:299] Negotiation successful
0806 22:51:38.449597 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 22:51:38.450300 5782 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.40.124" port: 40913 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:51:38.450686 5785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:38.450581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.124:40913 (local address 127.0.0.1:36594)
0806 22:51:38.450630 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0806 22:51:38.450664 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.124:40913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260806 22:51:38.451079 5785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:38.450998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.124:40913 (local address 127.0.0.1:36608)
0806 22:51:38.451041 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0806 22:51:38.451064 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.124:40913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260806 22:51:38.451200 5782 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.40.124:40913: Network error: Client connection negotiation failed: client connection to 127.6.40.124:40913: connect: Connection refused (error 111)
I20260806 22:51:38.457267 5887 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.40.124" port: 40913 } attempt: 1
I20260806 22:51:38.457800 5892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:38.457683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.124:40913 (local address 127.0.0.1:36612)
0806 22:51:38.457736 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0806 22:51:38.457764 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.124:40913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260806 22:51:38.458446 5892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:38.458270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.124:40913 (local address 127.0.0.1:36620)
0806 22:51:38.458376 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0806 22:51:38.458402 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.124:40913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
W20260806 22:51:38.458621 5887 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.40.124:40913: Network error: Client connection negotiation failed: client connection to 127.6.40.124:40913: connect: Connection refused (error 111)
I20260806 22:51:38.482338 5782 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.40.124" port: 40913 } attempt: 1
I20260806 22:51:38.482896 5785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:38.482719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.124:40913 (local address 127.0.0.1:36630)
0806 22:51:38.482788 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0806 22:51:38.482855 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.124:40913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260806 22:51:38.483399 5785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:38.483264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.124:40913 (local address 127.0.0.1:36644)
0806 22:51:38.483296 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0806 22:51:38.483321 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.124:40913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
W20260806 22:51:38.483577 5782 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.40.124:40913: Network error: Client connection negotiation failed: client connection to 127.6.40.124:40913: connect: Connection refused (error 111)
I20260806 22:51:38.526768 5887 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.40.124" port: 40913 } attempt: 2
I20260806 22:51:38.527242 5892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:38.527101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.124:40913 (local address 127.0.0.1:36654)
0806 22:51:38.527169 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0806 22:51:38.527210 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.124:40913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260806 22:51:38.527648 5892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:38.527550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.124:40913 (local address 127.0.0.1:36668)
0806 22:51:38.527596 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0806 22:51:38.527622 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.124:40913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260806 22:51:38.527742 5887 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.40.124:40913: Network error: Client connection negotiation failed: client connection to 127.6.40.124:40913: connect: Connection refused (error 111)
W20260806 22:51:38.530691 5897 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 22:51:38.530947 5897 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:51:38.531011 5897 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:51:38.531064 5897 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:51:38.532362 5897 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:51:38.532431 5897 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:51:38.532461 5897 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:51:38.532501 5897 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:51:38.532526 5897 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:51:38.534363 5897 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.40.84:37945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056405724609-6305-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056405724609-6305-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.6.40.126:41601,127.6.40.125:46517,127.6.40.124:40913
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056405724609-6305-0/krb5kdc/kudu_127.6.40.124.keytab
--principal=kudu/127.6.40.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.40.124:40913
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056405724609-6305-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.6.40.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5897
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056405724609-6305-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:51:38.534680 5897 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:51:38.534931 5897 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:51:38.540735 5908 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:51:38.540715 5903 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:51:38.540715 5904 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:51:38.541069 5897 server_base.cc:1034] running on GCE node
I20260806 22:51:38.541350 5897 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:51:38.541666 5897 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:51:38.542840 5897 hybrid_clock.cc:648] HybridClock initialized: now 1786056698542826 us; error 45 us; skew 500 ppm
Aug 06 22:51:38 dist-test-slave-zh2d krb5kdc[5670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056698, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:51:38.545533 5897 init.cc:377] Logged in from keytab as kudu/127.6.40.124@KRBTEST.COM (short username kudu)
I20260806 22:51:38.545982 5897 webserver.cc:460] Webserver started at http://127.6.40.124:40271/ using document root <none> and password file <none>
I20260806 22:51:38.546245 5897 fs_manager.cc:359] Metadata directory not provided
I20260806 22:51:38.546319 5897 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:51:38.546465 5897 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:51:38.547541 5897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056405724609-6305-0/minicluster-data/master-2/data/instance:
uuid: "dfee5cc89dbf4564871d95de8a365ec2"
format_stamp: "Formatted at 2026-08-06 22:51:38 on dist-test-slave-zh2d"
server_key: "2bb2e31bc59ceb3e5c4178564ecc9770"
server_key_iv: "c181da2a26da1a1bbdfcd15bb380a2a8"
server_key_version: "encryptionkey@0"
I20260806 22:51:38.547920 5897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056405724609-6305-0/minicluster-data/master-2/wal/instance:
uuid: "dfee5cc89dbf4564871d95de8a365ec2"
format_stamp: "Formatted at 2026-08-06 22:51:38 on dist-test-slave-zh2d"
server_key: "2bb2e31bc59ceb3e5c4178564ecc9770"
server_key_iv: "c181da2a26da1a1bbdfcd15bb380a2a8"
server_key_version: "encryptionkey@0"
I20260806 22:51:38.549347 5897 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:51:38.550197 5921 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:51:38.550434 5897 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:51:38.550549 5897 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056405724609-6305-0/minicluster-data/master-2/data,/tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056405724609-6305-0/minicluster-data/master-2/wal
uuid: "dfee5cc89dbf4564871d95de8a365ec2"
format_stamp: "Formatted at 2026-08-06 22:51:38 on dist-test-slave-zh2d"
server_key: "2bb2e31bc59ceb3e5c4178564ecc9770"
server_key_iv: "c181da2a26da1a1bbdfcd15bb380a2a8"
server_key_version: "encryptionkey@0"
I20260806 22:51:38.550635 5897 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056405724609-6305-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056405724609-6305-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056405724609-6305-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:51:38.558717 5782 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.40.124" port: 40913 } attempt: 2
I20260806 22:51:38.559152 5785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:38.559021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.124:40913 (local address 127.0.0.1:36676)
0806 22:51:38.559082 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0806 22:51:38.559123 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.124:40913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260806 22:51:38.559566 5785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:38.559477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.124:40913 (local address 127.0.0.1:36680)
0806 22:51:38.559523 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0806 22:51:38.559548 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.40.124:40913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260806 22:51:38.559719 5782 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.40.124:40913: Network error: Client connection negotiation failed: client connection to 127.6.40.124:40913: connect: Connection refused (error 111)
I20260806 22:51:38.575430 5897 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:51:38.577889 5897 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:51:38.578054 5897 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:51:38.582403 5897 rpc_server.cc:307] RPC server started. Bound to: 127.6.40.124:40913
I20260806 22:51:38.582458 5989 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.40.124:40913 every 8 connection(s)
I20260806 22:51:38.582801 5897 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056405724609-6305-0/minicluster-data/master-2/data/info.pb
I20260806 22:51:38.583873 5990 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:51:38.585572 5990 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.40.126" port: 41601 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:51:38.590407 6305 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_hhfSb/build/release/bin/kudu as pid 5897
I20260806 22:51:38.590545 6305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_hhfSb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056405724609-6305-0/minicluster-data/master-2/wal/instance
I20260806 22:51:38.590792 6305 external_mini_cluster.cc:1428] Setting key 0198c931efb6c114766b527c64e6bd5a
Aug 06 22:51:38 dist-test-slave-zh2d krb5kdc[5670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056698, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.124@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
I20260806 22:51:38.595821 5786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:38.586533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50858 (local address 127.6.40.126:41601)
0806 22:51:38.586654 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:38.586657 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:38.586853 (+ 196us) server_negotiation.cc:407] Connection header received
0806 22:51:38.587458 (+ 605us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:38.587460 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:38.587472 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:38.587533 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:38.588200 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:38.588678 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:38.589764 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:38.589964 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:38.590083 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:38.592646 (+ 2563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:51:38.592647 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:51:38.592648 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:51:38.592654 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:51:38.594238 (+ 1584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:38.594716 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:38.594717 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:38.594718 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:38.594770 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:38.595059 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:38.595060 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:38.595061 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:38.595216 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:51:38.595271 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:38.595625 (+ 354us) server_negotiation.cc:299] Negotiation successful
0806 22:51:38.595705 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 22:51:38.595834 5994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:38.586490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.126:41601 (local address 127.0.0.1:50858)
0806 22:51:38.586720 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0806 22:51:38.586770 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 22:51:38.587306 (+ 536us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:51:38.587614 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:51:38.587618 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:51:38.587893 (+ 275us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:51:38.588091 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:38.588095 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:38.588775 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:38.588776 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:38.589652 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:38.589655 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:38.590117 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:38.590119 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:38.590245 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:38.590793 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:51:38.590801 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:51:38.592552 (+ 1751us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:51:38.594322 (+ 1770us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:51:38.594329 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:51:38.594330 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:51:38.594639 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:51:38.594837 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:51:38.594838 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:51:38.594838 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:51:38.594984 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:51:38.595286 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:51:38.595293 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:51:38.595461 (+ 168us) client_negotiation.cc:769] Sending connection context
0806 22:51:38.595563 (+ 102us) client_negotiation.cc:240] Negotiation successful
0806 22:51:38.595676 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":94,"threads_started":1}
I20260806 22:51:38.596464 5990 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.40.125" port: 46517 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 22:51:38 dist-test-slave-zh2d krb5kdc[5670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056698, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.40.124@KRBTEST.COM
I20260806 22:51:38.600886 5997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:38.591201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36688 (local address 127.6.40.124:40913)
0806 22:51:38.591349 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:38.591360 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:38.591408 (+ 48us) server_negotiation.cc:407] Connection header received
0806 22:51:38.591479 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:38.591481 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:38.591498 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:38.591604 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:38.592256 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:38.593296 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:38.594403 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:38.594878 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:38.594948 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:38.597597 (+ 2649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:51:38.597606 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:51:38.597607 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:51:38.597614 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:51:38.599328 (+ 1714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:38.599758 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:38.599762 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:38.599762 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:38.599824 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:38.600226 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:38.600227 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:38.600228 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:38.600391 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:51:38.600570 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:38.600707 (+ 137us) server_negotiation.cc:299] Negotiation successful
0806 22:51:38.600768 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":73,"threads_started":1}
Aug 06 22:51:38 dist-test-slave-zh2d krb5kdc[5670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056698, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.124@KRBTEST.COM for kudu/127.6.40.125@KRBTEST.COM
I20260806 22:51:38.604159 5893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:38.596827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55244 (local address 127.6.40.125:46517)
0806 22:51:38.597216 (+ 389us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:38.597218 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:38.597227 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:51:38.597250 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:38.597251 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:38.597259 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:38.597331 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:38.597686 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:38.598290 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:38.599287 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:38.599597 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:38.599673 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:38.601765 (+ 2092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:51:38.601767 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:51:38.601768 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:51:38.601775 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:51:38.603097 (+ 1322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:38.603434 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:38.603435 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:38.603436 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:38.603498 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:38.603675 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:38.603676 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:38.603676 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:38.603803 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:51:38.603874 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:38.603999 (+ 125us) server_negotiation.cc:299] Negotiation successful
0806 22:51:38.604085 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362}
I20260806 22:51:38.604161 5994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:38.596747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.125:46517 (local address 127.0.0.1:55244)
0806 22:51:38.596816 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0806 22:51:38.596828 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:51:38.596904 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:51:38.597388 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:51:38.597389 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:51:38.597524 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:51:38.597598 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:38.597599 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:38.598408 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:38.598409 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:38.599195 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:38.599198 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:38.599676 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:38.599677 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:38.599755 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:38.600080 (+ 325us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:51:38.600084 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:51:38.601645 (+ 1561us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:51:38.603178 (+ 1533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:51:38.603180 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:51:38.603180 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:51:38.603365 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:51:38.603567 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:51:38.603568 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:51:38.603568 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:51:38.603616 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:51:38.603863 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:51:38.603863 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:51:38.603913 (+ 50us) client_negotiation.cc:769] Sending connection context
0806 22:51:38.603964 (+ 51us) client_negotiation.cc:240] Negotiation successful
0806 22:51:38.604066 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260806 22:51:38.604540 5990 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.40.124" port: 40913 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 22:51:38 dist-test-slave-zh2d krb5kdc[5670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056698, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.40.126@KRBTEST.COM
Aug 06 22:51:38 dist-test-slave-zh2d krb5kdc[5670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056698, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.124@KRBTEST.COM for kudu/127.6.40.124@KRBTEST.COM
I20260806 22:51:38.609817 5786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:38.601700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50874 (local address 127.6.40.126:41601)
0806 22:51:38.601743 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:38.601745 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:38.601792 (+ 47us) server_negotiation.cc:407] Connection header received
0806 22:51:38.601863 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:38.601864 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:38.601873 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:38.601911 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:38.602678 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:38.603122 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:38.604079 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:38.604375 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:38.604482 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:38.606919 (+ 2437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:51:38.606920 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:51:38.606920 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:51:38.606926 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:51:38.608450 (+ 1524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:38.608920 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:38.608921 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:38.608922 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:38.608978 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:38.609248 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:38.609250 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:38.609250 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:38.609387 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:51:38.609595 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:38.609635 (+ 40us) server_negotiation.cc:299] Negotiation successful
0806 22:51:38.609712 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 22:51:38.612526 5994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:38.605021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.124:40913 (local address 127.0.0.1:36698)
0806 22:51:38.605764 (+ 743us) negotiation.cc:107] Waiting for socket to connect
0806 22:51:38.605773 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:51:38.605815 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:51:38.605961 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:51:38.605962 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:51:38.606065 (+ 103us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:51:38.606094 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:38.606095 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:38.606987 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:38.606988 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:38.607729 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:38.607732 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:38.608175 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:38.608176 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:38.608241 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:38.608674 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:51:38.608677 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:51:38.609915 (+ 1238us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:51:38.611558 (+ 1643us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:51:38.611559 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:51:38.611560 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:51:38.611712 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:51:38.611906 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:51:38.611906 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:51:38.611907 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:51:38.611949 (+ 42us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:51:38.612207 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:51:38.612208 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:51:38.612249 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 22:51:38.612321 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 22:51:38.612415 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":726}
I20260806 22:51:38.612532 5997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:38.605047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36698 (local address 127.6.40.124:40913)
0806 22:51:38.605100 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:38.605102 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:38.605837 (+ 735us) server_negotiation.cc:407] Connection header received
0806 22:51:38.605865 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:38.605866 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:38.605879 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:38.605960 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:38.606165 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:38.606918 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:38.607847 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:38.608096 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:38.608152 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:38.609993 (+ 1841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:51:38.609994 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:51:38.609994 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:51:38.610001 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:51:38.611488 (+ 1487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:38.611781 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:38.611782 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:38.611783 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:38.611842 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:38.611985 (+ 143us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:38.611985 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:38.611985 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:38.612111 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:51:38.612184 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:38.612338 (+ 154us) server_negotiation.cc:299] Negotiation successful
0806 22:51:38.612458 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":57,"server-negotiator.queue_time_us":36}
I20260806 22:51:38.614307 5990 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dfee5cc89dbf4564871d95de8a365ec2: Bootstrap starting.
I20260806 22:51:38.615183 5990 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dfee5cc89dbf4564871d95de8a365ec2: Neither blocks nor log segments found. Creating new log.
Aug 06 22:51:38 dist-test-slave-zh2d krb5kdc[5670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056698, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.40.125@KRBTEST.COM
I20260806 22:51:38.615594 5990 log.cc:824] T 00000000000000000000000000000000 P dfee5cc89dbf4564871d95de8a365ec2: Log is configured to *not* fsync() on all Append() calls
I20260806 22:51:38.616701 5990 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dfee5cc89dbf4564871d95de8a365ec2: No bootstrap required, opened a new log
I20260806 22:51:38.618352 5990 raft_consensus.cc:348] T 00000000000000000000000000000000 P dfee5cc89dbf4564871d95de8a365ec2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbc514d3a698458fbf0eb2f5e304d5ef" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 41601 } } peers { permanent_uuid: "704c8f7e575f4354ac9d0bb5a6e7e432" member_type: VOTER last_known_addr { host: "127.6.40.125" port: 46517 } } peers { permanent_uuid: "dfee5cc89dbf4564871d95de8a365ec2" member_type: VOTER last_known_addr { host: "127.6.40.124" port: 40913 } }
I20260806 22:51:38.618489 5990 raft_consensus.cc:729] T 00000000000000000000000000000000 P dfee5cc89dbf4564871d95de8a365ec2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dfee5cc89dbf4564871d95de8a365ec2, State: Initialized, Role: FOLLOWER
I20260806 22:51:38.618546 5990 consensus_queue.cc:260] T 00000000000000000000000000000000 P dfee5cc89dbf4564871d95de8a365ec2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbc514d3a698458fbf0eb2f5e304d5ef" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 41601 } } peers { permanent_uuid: "704c8f7e575f4354ac9d0bb5a6e7e432" member_type: VOTER last_known_addr { host: "127.6.40.125" port: 46517 } } peers { permanent_uuid: "dfee5cc89dbf4564871d95de8a365ec2" member_type: VOTER last_known_addr { host: "127.6.40.124" port: 40913 } }
I20260806 22:51:38.618855 5990 sys_catalog.cc:564] T 00000000000000000000000000000000 P dfee5cc89dbf4564871d95de8a365ec2 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:51:38.618803 6003 sys_catalog.cc:455] T 00000000000000000000000000000000 P dfee5cc89dbf4564871d95de8a365ec2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbc514d3a698458fbf0eb2f5e304d5ef" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 41601 } } peers { permanent_uuid: "704c8f7e575f4354ac9d0bb5a6e7e432" member_type: VOTER last_known_addr { host: "127.6.40.125" port: 46517 } } peers { permanent_uuid: "dfee5cc89dbf4564871d95de8a365ec2" member_type: VOTER last_known_addr { host: "127.6.40.124" port: 40913 } } }
I20260806 22:51:38.618904 6003 sys_catalog.cc:458] T 00000000000000000000000000000000 P dfee5cc89dbf4564871d95de8a365ec2 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:51:38.618886 5887 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.40.124" port: 40913 } attempt: 3
I20260806 22:51:38.620234 5893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:38.610519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55254 (local address 127.6.40.125:46517)
0806 22:51:38.610566 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:38.610569 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:38.610841 (+ 272us) server_negotiation.cc:407] Connection header received
0806 22:51:38.610914 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:38.610915 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:38.610927 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:38.610982 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:38.611427 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:38.612102 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:38.613011 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:38.613316 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:38.613394 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:38.615904 (+ 2510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:51:38.615905 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:51:38.615905 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:51:38.615912 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:51:38.618828 (+ 2916us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:38.619350 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:38.619351 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:38.619352 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:38.619413 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:38.619666 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:38.619667 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:38.619668 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:38.619806 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:51:38.620005 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:38.620049 (+ 44us) server_negotiation.cc:299] Negotiation successful
0806 22:51:38.620128 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
W20260806 22:51:38.621165 6018 catalog_manager.cc:1558] T 00000000000000000000000000000000 P dfee5cc89dbf4564871d95de8a365ec2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:51:38.621220 6018 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:51:38.621855 6305 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
Aug 06 22:51:38 dist-test-slave-zh2d krb5kdc[5670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056698, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.125@KRBTEST.COM for kudu/127.6.40.124@KRBTEST.COM
I20260806 22:51:38.621256 5897 master_runner.cc:186] Error getting master registration for 127.6.40.126:41601: OK, instance_id { permanent_uuid: "dbc514d3a698458fbf0eb2f5e304d5ef" instance_seqno: 1786056698213728 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 22:51:38.623831 5897 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 22:51:38.623116 5744 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "dbc514d3a698458fbf0eb2f5e304d5ef"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50874
I20260806 22:51:38.626516 5997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:38.619218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36700 (local address 127.6.40.124:40913)
0806 22:51:38.619269 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:38.619271 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:38.619280 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:51:38.619306 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:38.619307 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:38.619317 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:38.619782 (+ 465us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:38.619813 (+ 31us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:38.620534 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:38.621500 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:38.621777 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:38.623854 (+ 2077us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:38.623910 (+ 56us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:51:38.623911 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:51:38.623911 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:51:38.623915 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:51:38.625347 (+ 1432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:38.625719 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:38.625721 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:38.625721 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:38.625778 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:38.625982 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:38.625983 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:38.625984 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:38.626112 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:51:38.626191 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:38.626352 (+ 161us) server_negotiation.cc:299] Negotiation successful
0806 22:51:38.626429 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 22:51:38.626518 5892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:38.619145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.124:40913 (local address 127.0.0.1:36700)
0806 22:51:38.619201 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0806 22:51:38.619212 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:51:38.619253 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:51:38.619427 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:51:38.619429 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:51:38.619640 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:51:38.619693 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:38.619696 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:38.620642 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:38.620643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:38.621402 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:38.621405 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:38.621875 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:38.621876 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:38.621931 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:38.622349 (+ 418us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:51:38.622354 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:51:38.623767 (+ 1413us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:51:38.625454 (+ 1687us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:51:38.625455 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:51:38.625456 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:51:38.625631 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:51:38.625847 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:51:38.625848 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:51:38.625848 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:51:38.625907 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:51:38.626195 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:51:38.626196 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:51:38.626240 (+ 44us) client_negotiation.cc:769] Sending connection context
0806 22:51:38.626327 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 22:51:38.626428 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260806 22:51:38.629086 5887 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 704c8f7e575f4354ac9d0bb5a6e7e432: Bootstrap starting.
I20260806 22:51:38.629760 5887 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 704c8f7e575f4354ac9d0bb5a6e7e432: Neither blocks nor log segments found. Creating new log.
I20260806 22:51:38.630092 5887 log.cc:824] T 00000000000000000000000000000000 P 704c8f7e575f4354ac9d0bb5a6e7e432: Log is configured to *not* fsync() on all Append() calls
I20260806 22:51:38.631038 5887 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 704c8f7e575f4354ac9d0bb5a6e7e432: No bootstrap required, opened a new log
I20260806 22:51:38.633497 5887 raft_consensus.cc:348] T 00000000000000000000000000000000 P 704c8f7e575f4354ac9d0bb5a6e7e432 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbc514d3a698458fbf0eb2f5e304d5ef" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 41601 } } peers { permanent_uuid: "704c8f7e575f4354ac9d0bb5a6e7e432" member_type: VOTER last_known_addr { host: "127.6.40.125" port: 46517 } } peers { permanent_uuid: "dfee5cc89dbf4564871d95de8a365ec2" member_type: VOTER last_known_addr { host: "127.6.40.124" port: 40913 } }
I20260806 22:51:38.633659 5887 raft_consensus.cc:729] T 00000000000000000000000000000000 P 704c8f7e575f4354ac9d0bb5a6e7e432 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 704c8f7e575f4354ac9d0bb5a6e7e432, State: Initialized, Role: FOLLOWER
I20260806 22:51:38.633755 5887 consensus_queue.cc:260] T 00000000000000000000000000000000 P 704c8f7e575f4354ac9d0bb5a6e7e432 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbc514d3a698458fbf0eb2f5e304d5ef" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 41601 } } peers { permanent_uuid: "704c8f7e575f4354ac9d0bb5a6e7e432" member_type: VOTER last_known_addr { host: "127.6.40.125" port: 46517 } } peers { permanent_uuid: "dfee5cc89dbf4564871d95de8a365ec2" member_type: VOTER last_known_addr { host: "127.6.40.124" port: 40913 } }
I20260806 22:51:38.634076 6022 sys_catalog.cc:455] T 00000000000000000000000000000000 P 704c8f7e575f4354ac9d0bb5a6e7e432 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbc514d3a698458fbf0eb2f5e304d5ef" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 41601 } } peers { permanent_uuid: "704c8f7e575f4354ac9d0bb5a6e7e432" member_type: VOTER last_known_addr { host: "127.6.40.125" port: 46517 } } peers { permanent_uuid: "dfee5cc89dbf4564871d95de8a365ec2" member_type: VOTER last_known_addr { host: "127.6.40.124" port: 40913 } } }
I20260806 22:51:38.634145 5887 sys_catalog.cc:564] T 00000000000000000000000000000000 P 704c8f7e575f4354ac9d0bb5a6e7e432 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:51:38.634181 6022 sys_catalog.cc:458] T 00000000000000000000000000000000 P 704c8f7e575f4354ac9d0bb5a6e7e432 [sys.catalog]: This master's current role is: FOLLOWER
W20260806 22:51:38.636180 6038 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 704c8f7e575f4354ac9d0bb5a6e7e432: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:51:38.636235 6038 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:51:38.636534 5792 master_runner.cc:186] Error getting master registration for 127.6.40.126:41601: OK, instance_id { permanent_uuid: "dbc514d3a698458fbf0eb2f5e304d5ef" instance_seqno: 1786056698213728 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 22:51:38.639802 5792 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 22:51:38.658913 5782 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.40.124" port: 40913 } attempt: 3
Aug 06 22:51:38 dist-test-slave-zh2d krb5kdc[5670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056698, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.40.126@KRBTEST.COM for kudu/127.6.40.124@KRBTEST.COM
I20260806 22:51:38.668323 5785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:38.659317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.40.124:40913 (local address 127.0.0.1:36714)
0806 22:51:38.659448 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0806 22:51:38.659467 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 22:51:38.659546 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:51:38.659798 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:51:38.659799 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:51:38.660044 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:51:38.660104 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:38.660107 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:38.661209 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:38.661225 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:38.662079 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:38.662082 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:38.662594 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:38.662595 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:38.662661 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:38.663234 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:51:38.663240 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:51:38.664936 (+ 1696us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:51:38.667136 (+ 2200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:51:38.667137 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:51:38.667138 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:51:38.667374 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:51:38.667603 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:51:38.667604 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:51:38.667604 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:51:38.667667 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:51:38.667990 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:51:38.667991 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:51:38.668041 (+ 50us) client_negotiation.cc:769] Sending connection context
0806 22:51:38.668113 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 22:51:38.668216 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98}
I20260806 22:51:38.668360 5997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:38.659571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36714 (local address 127.6.40.124:40913)
0806 22:51:38.659627 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:38.659629 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:38.659639 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:51:38.659669 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:38.659670 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:38.659682 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:38.659741 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:38.660207 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:38.661092 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:38.662165 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:38.662505 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:38.662601 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:38.665029 (+ 2428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:51:38.665031 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:51:38.665031 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:51:38.665039 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:51:38.667058 (+ 2019us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:38.667460 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:38.667462 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:38.667462 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:38.667530 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:38.667734 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:38.667735 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:38.667736 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:38.667902 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:51:38.667987 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:38.668160 (+ 173us) server_negotiation.cc:299] Negotiation successful
0806 22:51:38.668246 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 22:51:38.670815 5782 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dbc514d3a698458fbf0eb2f5e304d5ef: Bootstrap starting.
I20260806 22:51:38.671684 5782 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dbc514d3a698458fbf0eb2f5e304d5ef: Neither blocks nor log segments found. Creating new log.
I20260806 22:51:38.672065 5782 log.cc:824] T 00000000000000000000000000000000 P dbc514d3a698458fbf0eb2f5e304d5ef: Log is configured to *not* fsync() on all Append() calls
I20260806 22:51:38.673137 5782 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dbc514d3a698458fbf0eb2f5e304d5ef: No bootstrap required, opened a new log
I20260806 22:51:38.674727 5782 raft_consensus.cc:348] T 00000000000000000000000000000000 P dbc514d3a698458fbf0eb2f5e304d5ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbc514d3a698458fbf0eb2f5e304d5ef" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 41601 } } peers { permanent_uuid: "704c8f7e575f4354ac9d0bb5a6e7e432" member_type: VOTER last_known_addr { host: "127.6.40.125" port: 46517 } } peers { permanent_uuid: "dfee5cc89dbf4564871d95de8a365ec2" member_type: VOTER last_known_addr { host: "127.6.40.124" port: 40913 } }
I20260806 22:51:38.674924 5782 raft_consensus.cc:729] T 00000000000000000000000000000000 P dbc514d3a698458fbf0eb2f5e304d5ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbc514d3a698458fbf0eb2f5e304d5ef, State: Initialized, Role: FOLLOWER
I20260806 22:51:38.675007 5782 consensus_queue.cc:260] T 00000000000000000000000000000000 P dbc514d3a698458fbf0eb2f5e304d5ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbc514d3a698458fbf0eb2f5e304d5ef" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 41601 } } peers { permanent_uuid: "704c8f7e575f4354ac9d0bb5a6e7e432" member_type: VOTER last_known_addr { host: "127.6.40.125" port: 46517 } } peers { permanent_uuid: "dfee5cc89dbf4564871d95de8a365ec2" member_type: VOTER last_known_addr { host: "127.6.40.124" port: 40913 } }
I20260806 22:51:38.675349 5782 sys_catalog.cc:564] T 00000000000000000000000000000000 P dbc514d3a698458fbf0eb2f5e304d5ef [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:51:38.675369 5744 raft_consensus.cc:482] T 00000000000000000000000000000000 P dbc514d3a698458fbf0eb2f5e304d5ef [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260806 22:51:38.675442 5744 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dbc514d3a698458fbf0eb2f5e304d5ef [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:51:38.676262 5744 raft_consensus.cc:504] T 00000000000000000000000000000000 P dbc514d3a698458fbf0eb2f5e304d5ef [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbc514d3a698458fbf0eb2f5e304d5ef" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 41601 } } peers { permanent_uuid: "704c8f7e575f4354ac9d0bb5a6e7e432" member_type: VOTER last_known_addr { host: "127.6.40.125" port: 46517 } } peers { permanent_uuid: "dfee5cc89dbf4564871d95de8a365ec2" member_type: VOTER last_known_addr { host: "127.6.40.124" port: 40913 } }
I20260806 22:51:38.676523 5744 leader_election.cc:288] T 00000000000000000000000000000000 P dbc514d3a698458fbf0eb2f5e304d5ef [CANDIDATE]: Term 1 election: Requested vote from peers 704c8f7e575f4354ac9d0bb5a6e7e432 (127.6.40.125:46517), dfee5cc89dbf4564871d95de8a365ec2 (127.6.40.124:40913)
I20260806 22:51:38.676905 5850 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "dbc514d3a698458fbf0eb2f5e304d5ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "704c8f7e575f4354ac9d0bb5a6e7e432"
I20260806 22:51:38.677062 5850 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 704c8f7e575f4354ac9d0bb5a6e7e432 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:51:38.678093 5850 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 704c8f7e575f4354ac9d0bb5a6e7e432 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbc514d3a698458fbf0eb2f5e304d5ef in term 1.
I20260806 22:51:38.677520 6042 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbc514d3a698458fbf0eb2f5e304d5ef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbc514d3a698458fbf0eb2f5e304d5ef" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 41601 } } peers { permanent_uuid: "704c8f7e575f4354ac9d0bb5a6e7e432" member_type: VOTER last_known_addr { host: "127.6.40.125" port: 46517 } } peers { permanent_uuid: "dfee5cc89dbf4564871d95de8a365ec2" member_type: VOTER last_known_addr { host: "127.6.40.124" port: 40913 } } }
I20260806 22:51:38.678853 6042 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbc514d3a698458fbf0eb2f5e304d5ef [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:51:38.678462 5718 leader_election.cc:302] T 00000000000000000000000000000000 P dbc514d3a698458fbf0eb2f5e304d5ef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 704c8f7e575f4354ac9d0bb5a6e7e432, dbc514d3a698458fbf0eb2f5e304d5ef; no voters:
I20260806 22:51:38.680042 6042 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dbc514d3a698458fbf0eb2f5e304d5ef [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:51:38.680189 6042 raft_consensus.cc:686] T 00000000000000000000000000000000 P dbc514d3a698458fbf0eb2f5e304d5ef [term 1 LEADER]: Becoming Leader. State: Replica: dbc514d3a698458fbf0eb2f5e304d5ef, State: Running, Role: LEADER
I20260806 22:51:38.680279 6042 consensus_queue.cc:237] T 00000000000000000000000000000000 P dbc514d3a698458fbf0eb2f5e304d5ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbc514d3a698458fbf0eb2f5e304d5ef" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 41601 } } peers { permanent_uuid: "704c8f7e575f4354ac9d0bb5a6e7e432" member_type: VOTER last_known_addr { host: "127.6.40.125" port: 46517 } } peers { permanent_uuid: "dfee5cc89dbf4564871d95de8a365ec2" member_type: VOTER last_known_addr { host: "127.6.40.124" port: 40913 } }
I20260806 22:51:38.680688 6042 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbc514d3a698458fbf0eb2f5e304d5ef [sys.catalog]: SysCatalogTable state changed. Reason: New leader dbc514d3a698458fbf0eb2f5e304d5ef. Latest consensus state: current_term: 1 leader_uuid: "dbc514d3a698458fbf0eb2f5e304d5ef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbc514d3a698458fbf0eb2f5e304d5ef" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 41601 } } peers { permanent_uuid: "704c8f7e575f4354ac9d0bb5a6e7e432" member_type: VOTER last_known_addr { host: "127.6.40.125" port: 46517 } } peers { permanent_uuid: "dfee5cc89dbf4564871d95de8a365ec2" member_type: VOTER last_known_addr { host: "127.6.40.124" port: 40913 } } }
I20260806 22:51:38.680759 6042 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbc514d3a698458fbf0eb2f5e304d5ef [sys.catalog]: This master's current role is: LEADER
I20260806 22:51:38.681151 5961 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "dbc514d3a698458fbf0eb2f5e304d5ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "dfee5cc89dbf4564871d95de8a365ec2"
I20260806 22:51:38.681314 5961 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dfee5cc89dbf4564871d95de8a365ec2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:51:38.682324 5961 raft_consensus.cc:2454] T 00000000000000000000000000000000 P dfee5cc89dbf4564871d95de8a365ec2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbc514d3a698458fbf0eb2f5e304d5ef in term 1.
I20260806 22:51:38.683590 5687 master_runner.cc:263] No leader master found from master dbc514d3a698458fbf0eb2f5e304d5ef
I20260806 22:51:38.683640 5687 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260806 22:51:38.684950 6063 catalog_manager.cc:1558] T 00000000000000000000000000000000 P dbc514d3a698458fbf0eb2f5e304d5ef: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:51:38.685086 6063 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:51:38.685634 6072 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:51:38.685848 6072 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:51:38.692062 5786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:38.679707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50890 (local address 127.6.40.126:41601)
0806 22:51:38.679749 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:38.679751 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:38.684056 (+ 4305us) server_negotiation.cc:407] Connection header received
0806 22:51:38.684158 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:38.684159 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:38.684170 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:38.684200 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:38.685058 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:38.685820 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:38.686916 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:38.687210 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:38.687251 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:38.688957 (+ 1706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:51:38.688959 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:51:38.688959 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:51:38.688966 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:51:38.690766 (+ 1800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:38.691195 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:38.691196 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:38.691196 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:38.691255 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:38.691467 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:38.691468 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:38.691468 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:38.691594 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:51:38.691691 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:38.691889 (+ 198us) server_negotiation.cc:299] Negotiation successful
0806 22:51:38.691967 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 22:51:38.694077 5961 raft_consensus.cc:1261] T 00000000000000000000000000000000 P dfee5cc89dbf4564871d95de8a365ec2 [term 1 FOLLOWER]: Refusing update from remote peer dbc514d3a698458fbf0eb2f5e304d5ef: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:51:38.694464 6042 consensus_queue.cc:1035] T 00000000000000000000000000000000 P dbc514d3a698458fbf0eb2f5e304d5ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "dfee5cc89dbf4564871d95de8a365ec2" member_type: VOTER last_known_addr { host: "127.6.40.124" port: 40913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:51:38.694942 5850 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 704c8f7e575f4354ac9d0bb5a6e7e432 [term 1 FOLLOWER]: Refusing update from remote peer dbc514d3a698458fbf0eb2f5e304d5ef: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:51:38.695250 6070 consensus_queue.cc:1035] T 00000000000000000000000000000000 P dbc514d3a698458fbf0eb2f5e304d5ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "704c8f7e575f4354ac9d0bb5a6e7e432" member_type: VOTER last_known_addr { host: "127.6.40.125" port: 46517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:51:38.697052 6022 sys_catalog.cc:455] T 00000000000000000000000000000000 P 704c8f7e575f4354ac9d0bb5a6e7e432 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dbc514d3a698458fbf0eb2f5e304d5ef. Latest consensus state: current_term: 1 leader_uuid: "dbc514d3a698458fbf0eb2f5e304d5ef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbc514d3a698458fbf0eb2f5e304d5ef" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 41601 } } peers { permanent_uuid: "704c8f7e575f4354ac9d0bb5a6e7e432" member_type: VOTER last_known_addr { host: "127.6.40.125" port: 46517 } } peers { permanent_uuid: "dfee5cc89dbf4564871d95de8a365ec2" member_type: VOTER last_known_addr { host: "127.6.40.124" port: 40913 } } }
I20260806 22:51:38.697175 6022 sys_catalog.cc:458] T 00000000000000000000000000000000 P 704c8f7e575f4354ac9d0bb5a6e7e432 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:51:38.697240 6003 sys_catalog.cc:455] T 00000000000000000000000000000000 P dfee5cc89dbf4564871d95de8a365ec2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dbc514d3a698458fbf0eb2f5e304d5ef. Latest consensus state: current_term: 1 leader_uuid: "dbc514d3a698458fbf0eb2f5e304d5ef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbc514d3a698458fbf0eb2f5e304d5ef" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 41601 } } peers { permanent_uuid: "704c8f7e575f4354ac9d0bb5a6e7e432" member_type: VOTER last_known_addr { host: "127.6.40.125" port: 46517 } } peers { permanent_uuid: "dfee5cc89dbf4564871d95de8a365ec2" member_type: VOTER last_known_addr { host: "127.6.40.124" port: 40913 } } }
I20260806 22:51:38.697371 6003 sys_catalog.cc:458] T 00000000000000000000000000000000 P dfee5cc89dbf4564871d95de8a365ec2 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:51:38.698355 5997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:38.683644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36722 (local address 127.6.40.124:40913)
0806 22:51:38.683738 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:38.683740 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:38.685313 (+ 1573us) server_negotiation.cc:407] Connection header received
0806 22:51:38.685429 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:38.685431 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:38.685444 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:38.685477 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:38.686478 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:38.687295 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:38.688827 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:38.689510 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:38.689558 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:38.691389 (+ 1831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:51:38.691391 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:51:38.691391 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:51:38.691399 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:51:38.693583 (+ 2184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:38.694046 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:38.694048 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:38.694048 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:38.694115 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:38.696310 (+ 2195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:38.696311 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:38.696312 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:38.696498 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:51:38.696605 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:38.697354 (+ 749us) server_negotiation.cc:299] Negotiation successful
0806 22:51:38.697925 (+ 571us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 22:51:38.700134 5893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:38.684097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55266 (local address 127.6.40.125:46517)
0806 22:51:38.684139 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:38.684141 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:38.692793 (+ 8652us) server_negotiation.cc:407] Connection header received
0806 22:51:38.692958 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:38.692960 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:38.692975 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:38.693008 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:38.693586 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:38.694275 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:38.695308 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:38.695581 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:38.695621 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:38.697098 (+ 1477us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:51:38.697099 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:51:38.697100 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:51:38.697109 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:51:38.698880 (+ 1771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:38.699271 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:38.699272 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:38.699272 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:38.699328 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:38.699547 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:38.699548 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:38.699548 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:38.699676 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:51:38.699761 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:38.699964 (+ 203us) server_negotiation.cc:299] Negotiation successful
0806 22:51:38.700046 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 22:51:38.701102 6042 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbc514d3a698458fbf0eb2f5e304d5ef [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "dbc514d3a698458fbf0eb2f5e304d5ef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbc514d3a698458fbf0eb2f5e304d5ef" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 41601 } } peers { permanent_uuid: "704c8f7e575f4354ac9d0bb5a6e7e432" member_type: VOTER last_known_addr { host: "127.6.40.125" port: 46517 } } peers { permanent_uuid: "dfee5cc89dbf4564871d95de8a365ec2" member_type: VOTER last_known_addr { host: "127.6.40.124" port: 40913 } } }
I20260806 22:51:38.701222 6042 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbc514d3a698458fbf0eb2f5e304d5ef [sys.catalog]: This master's current role is: LEADER
I20260806 22:51:38.701259 6072 catalog_manager.cc:1347] Generated new cluster ID: 47f846a9f9944dca8ab71336b1bffe14
I20260806 22:51:38.701308 6072 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:51:38.701615 6042 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbc514d3a698458fbf0eb2f5e304d5ef [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "dbc514d3a698458fbf0eb2f5e304d5ef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbc514d3a698458fbf0eb2f5e304d5ef" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 41601 } } peers { permanent_uuid: "704c8f7e575f4354ac9d0bb5a6e7e432" member_type: VOTER last_known_addr { host: "127.6.40.125" port: 46517 } } peers { permanent_uuid: "dfee5cc89dbf4564871d95de8a365ec2" member_type: VOTER last_known_addr { host: "127.6.40.124" port: 40913 } } }
I20260806 22:51:38.701700 6042 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbc514d3a698458fbf0eb2f5e304d5ef [sys.catalog]: This master's current role is: LEADER
I20260806 22:51:38.701927 6003 sys_catalog.cc:455] T 00000000000000000000000000000000 P dfee5cc89dbf4564871d95de8a365ec2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "dbc514d3a698458fbf0eb2f5e304d5ef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbc514d3a698458fbf0eb2f5e304d5ef" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 41601 } } peers { permanent_uuid: "704c8f7e575f4354ac9d0bb5a6e7e432" member_type: VOTER last_known_addr { host: "127.6.40.125" port: 46517 } } peers { permanent_uuid: "dfee5cc89dbf4564871d95de8a365ec2" member_type: VOTER last_known_addr { host: "127.6.40.124" port: 40913 } } }
I20260806 22:51:38.702010 6003 sys_catalog.cc:458] T 00000000000000000000000000000000 P dfee5cc89dbf4564871d95de8a365ec2 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:51:38.702109 6022 sys_catalog.cc:455] T 00000000000000000000000000000000 P 704c8f7e575f4354ac9d0bb5a6e7e432 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "dbc514d3a698458fbf0eb2f5e304d5ef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbc514d3a698458fbf0eb2f5e304d5ef" member_type: VOTER last_known_addr { host: "127.6.40.126" port: 41601 } } peers { permanent_uuid: "704c8f7e575f4354ac9d0bb5a6e7e432" member_type: VOTER last_known_addr { host: "127.6.40.125" port: 46517 } } peers { permanent_uuid: "dfee5cc89dbf4564871d95de8a365ec2" member_type: VOTER last_known_addr { host: "127.6.40.124" port: 40913 } } }
I20260806 22:51:38.702194 6022 sys_catalog.cc:458] T 00000000000000000000000000000000 P 704c8f7e575f4354ac9d0bb5a6e7e432 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:51:38.702567 6078 mvcc.cc:205] Tried to move back new op lower bound from 7315688237847457792 to 7315688237795082240. Current Snapshot: MvccSnapshot[applied={T|T < 7315688237847457792}]
I20260806 22:51:38.702726 6077 mvcc.cc:205] Tried to move back new op lower bound from 7315688237847457792 to 7315688237795082240. Current Snapshot: MvccSnapshot[applied={T|T < 7315688237847457792 or (T in {7315688237847457792})}]
I20260806 22:51:38.702863 6081 mvcc.cc:205] Tried to move back new op lower bound from 7315688237847457792 to 7315688237795082240. Current Snapshot: MvccSnapshot[applied={T|T < 7315688237847457792}]
I20260806 22:51:38.710999 6072 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:51:38.711716 6072 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:51:38.715466 6072 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dbc514d3a698458fbf0eb2f5e304d5ef: Generated new TSK 0
I20260806 22:51:38.715647 6072 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 22:51:38.738652 5786 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:51:38.737106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50892 (local address 127.6.40.126:41601)
0806 22:51:38.737166 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:38.737169 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:38.737905 (+ 736us) server_negotiation.cc:407] Connection header received
0806 22:51:38.738029 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:38.738030 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:38.738042 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:38.738076 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:38.738578 (+ 502us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50892: BlockingRecv error: recv got EOF from 127.0.0.1:50892 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260806 22:51:38.739564 5893 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:51:38.737575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55278 (local address 127.6.40.125:46517)
0806 22:51:38.737618 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:38.737620 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:38.738884 (+ 1264us) server_negotiation.cc:407] Connection header received
0806 22:51:38.739072 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:38.739073 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:38.739085 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:38.739118 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:38.739500 (+ 382us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55278: BlockingRecv error: recv got EOF from 127.0.0.1:55278 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260806 22:51:38.740650 5997 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:51:38.739932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36734 (local address 127.6.40.124:40913)
0806 22:51:38.739980 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:38.739982 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:38.740086 (+ 104us) server_negotiation.cc:407] Connection header received
0806 22:51:38.740187 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:38.740188 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:38.740198 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:38.740240 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:38.740585 (+ 345us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36734: BlockingRecv error: recv got EOF from 127.0.0.1:36734 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260806 22:51:38.745980 5893 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:51:38.745153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55292 (local address 127.6.40.125:46517)
0806 22:51:38.745194 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:38.745196 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:38.745415 (+ 219us) server_negotiation.cc:407] Connection header received
0806 22:51:38.745531 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:38.745532 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:38.745551 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:38.745583 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:38.745925 (+ 342us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55292: BlockingRecv error: recv got EOF from 127.0.0.1:55292 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260806 22:51:38.749178 5893 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:51:38.748314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55308 (local address 127.6.40.125:46517)
0806 22:51:38.748363 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:38.748364 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:38.748628 (+ 264us) server_negotiation.cc:407] Connection header received
0806 22:51:38.748732 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:38.748733 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:38.748743 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:38.748775 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:38.749131 (+ 356us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55308: BlockingRecv error: recv got EOF from 127.0.0.1:55308 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
W20260806 22:51:38.753253 5893 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:51:38.752299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55320 (local address 127.6.40.125:46517)
0806 22:51:38.752347 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:38.752349 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:38.752613 (+ 264us) server_negotiation.cc:407] Connection header received
0806 22:51:38.752715 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:38.752716 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:38.752726 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:38.752758 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:38.753205 (+ 447us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55320: BlockingRecv error: recv got EOF from 127.0.0.1:55320 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260806 22:51:38.759160 5893 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:51:38.758360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55330 (local address 127.6.40.125:46517)
0806 22:51:38.758395 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:38.758397 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:38.758644 (+ 247us) server_negotiation.cc:407] Connection header received
0806 22:51:38.758742 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:38.758743 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:38.758753 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:38.758784 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:38.759096 (+ 312us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55330: BlockingRecv error: recv got EOF from 127.0.0.1:55330 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260806 22:51:38.769462 5893 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:51:38.768451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55332 (local address 127.6.40.125:46517)
0806 22:51:38.768493 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:38.768495 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:38.768779 (+ 284us) server_negotiation.cc:407] Connection header received
0806 22:51:38.768972 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:38.768973 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:38.768985 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:38.769018 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:38.769403 (+ 385us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55332: BlockingRecv error: recv got EOF from 127.0.0.1:55332 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260806 22:51:38.787639 5893 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:51:38.786704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55340 (local address 127.6.40.125:46517)
0806 22:51:38.786748 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:38.786750 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:38.787054 (+ 304us) server_negotiation.cc:407] Connection header received
0806 22:51:38.787159 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:38.787160 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:38.787172 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:38.787202 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:38.787579 (+ 377us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55340: BlockingRecv error: recv got EOF from 127.0.0.1:55340 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260806 22:51:38.822144 5893 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:51:38.821092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55344 (local address 127.6.40.125:46517)
0806 22:51:38.821145 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:38.821148 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:38.821488 (+ 340us) server_negotiation.cc:407] Connection header received
0806 22:51:38.821604 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:38.821605 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:38.821618 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:38.821651 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:38.822078 (+ 427us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55344: BlockingRecv error: recv got EOF from 127.0.0.1:55344 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260806 22:51:38.888312 5893 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:51:38.887589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55354 (local address 127.6.40.125:46517)
0806 22:51:38.887657 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:38.887660 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:38.887743 (+ 83us) server_negotiation.cc:407] Connection header received
0806 22:51:38.887814 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:38.887815 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:38.887826 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:38.887872 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:38.888249 (+ 377us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55354: BlockingRecv error: recv got EOF from 127.0.0.1:55354 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260806 22:51:39.018597 5893 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:51:39.017799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55368 (local address 127.6.40.125:46517)
0806 22:51:39.017879 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:39.017882 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:39.017968 (+ 86us) server_negotiation.cc:407] Connection header received
0806 22:51:39.018025 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:39.018026 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:39.018037 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:39.018093 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:39.018513 (+ 420us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55368: BlockingRecv error: recv got EOF from 127.0.0.1:55368 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260806 22:51:39.276927 5893 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:51:39.275943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55370 (local address 127.6.40.125:46517)
0806 22:51:39.276005 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:39.276008 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:39.276253 (+ 245us) server_negotiation.cc:407] Connection header received
0806 22:51:39.276330 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:39.276331 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:39.276345 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:39.276416 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:39.276856 (+ 440us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55370: BlockingRecv error: recv got EOF from 127.0.0.1:55370 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 22:51:39.621682 6018 catalog_manager.cc:1259] Loaded cluster ID: 47f846a9f9944dca8ab71336b1bffe14
I20260806 22:51:39.621768 6018 catalog_manager.cc:1552] T 00000000000000000000000000000000 P dfee5cc89dbf4564871d95de8a365ec2: loading cluster ID for follower catalog manager: success
I20260806 22:51:39.622622 6018 catalog_manager.cc:1574] T 00000000000000000000000000000000 P dfee5cc89dbf4564871d95de8a365ec2: acquiring CA information for follower catalog manager: success
I20260806 22:51:39.623020 6018 catalog_manager.cc:1602] T 00000000000000000000000000000000 P dfee5cc89dbf4564871d95de8a365ec2: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 22:51:39.636687 6038 catalog_manager.cc:1259] Loaded cluster ID: 47f846a9f9944dca8ab71336b1bffe14
I20260806 22:51:39.636776 6038 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 704c8f7e575f4354ac9d0bb5a6e7e432: loading cluster ID for follower catalog manager: success
I20260806 22:51:39.637645 6038 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 704c8f7e575f4354ac9d0bb5a6e7e432: acquiring CA information for follower catalog manager: success
I20260806 22:51:39.637997 6038 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 704c8f7e575f4354ac9d0bb5a6e7e432: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 22:51:39.793027 6182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:39.790140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55374 (local address 127.6.40.125:46517)
0806 22:51:39.790259 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:39.790261 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:39.790315 (+ 54us) server_negotiation.cc:407] Connection header received
0806 22:51:39.790364 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:39.790365 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:39.790432 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:39.790522 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:51:39.790703 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:39.791284 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:39.792139 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:39.792342 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:39.792405 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:39.792609 (+ 204us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:51:39.792724 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:51:39.792787 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:39.792817 (+ 30us) server_negotiation.cc:299] Negotiation successful
0806 22:51:39.792892 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":69,"threads_started":1}
I20260806 22:51:39.797710 6189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:39.795045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36746 (local address 127.6.40.124:40913)
0806 22:51:39.795182 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:39.795185 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:39.795195 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:51:39.795246 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:39.795247 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:39.795286 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:39.795379 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:51:39.795509 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:39.796016 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:39.796800 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:39.797081 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:39.797182 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:39.797352 (+ 170us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:51:39.797462 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:51:39.797545 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:39.797586 (+ 41us) server_negotiation.cc:299] Negotiation successful
0806 22:51:39.797609 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":70,"threads_started":1}
I20260806 22:51:39.802393 6195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:39.799386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50898 (local address 127.6.40.126:41601)
0806 22:51:39.799735 (+ 349us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:39.799738 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:39.799746 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:51:39.799791 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:39.799792 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:39.799876 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:39.799960 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:51:39.800081 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:39.800620 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:39.801518 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:39.801765 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:39.801862 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:39.802053 (+ 191us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:51:39.802141 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:51:39.802210 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:39.802279 (+ 69us) server_negotiation.cc:299] Negotiation successful
0806 22:51:39.802303 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":51,"threads_started":1}
I20260806 22:51:39.803220 6182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:39.799336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55388 (local address 127.6.40.125:46517)
0806 22:51:39.799507 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:39.799509 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:39.799614 (+ 105us) server_negotiation.cc:407] Connection header received
0806 22:51:39.799680 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:39.799681 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:39.799694 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:39.799761 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:51:39.799914 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:39.800693 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:39.802365 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:39.802666 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:39.802708 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:39.802929 (+ 221us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:51:39.802962 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:51:39.803018 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:39.803101 (+ 83us) server_negotiation.cc:299] Negotiation successful
0806 22:51:39.803129 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147}
I20260806 22:51:39.803486 6189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:39.800573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36754 (local address 127.6.40.124:40913)
0806 22:51:39.800648 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:39.800650 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:39.800699 (+ 49us) server_negotiation.cc:407] Connection header received
0806 22:51:39.800809 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:39.800810 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:39.800821 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:39.800905 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:51:39.801031 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:39.801498 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:39.802485 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:39.802702 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:39.802747 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:39.803198 (+ 451us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:51:39.803235 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:51:39.803264 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:39.803392 (+ 128us) server_negotiation.cc:299] Negotiation successful
0806 22:51:39.803415 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 22:51:39.804716 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 5687
I20260806 22:51:39.818537 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 5792
I20260806 22:51:39.829018 6305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_hhfSb/build/release/bin/kudu with pid 5897
2026-08-06T22:51:39Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3176 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6179 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (294119 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST